-- Logs begin at Tue 2022-07-12 02:11:58 UTC, end at Tue 2022-07-12 04:13:56 UTC. -- Jul 12 03:34:48.474344 dv-cinder-847730-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 12 03:34:48.481860 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 12 03:34:48 2022] *** Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: nodename: dv-cinder-847730-4-iscsi Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: machine: x86_64 Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: clock source: unix Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: pcre jit disabled Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: detected number of CPU cores: 4 Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: current working directory: / Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: detected binary path: /usr/bin/uwsgi-core Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: your processes number limit is 47680 Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: your memory page size is 4096 bytes Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: detected max file descriptor number: 2048 Jul 12 03:34:48.483909 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: lock engine: pthread robust mutexes Jul 12 03:34:48.485474 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: thunder lock: enabled Jul 12 03:34:48.485474 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 12 03:34:48.485474 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 12 03:34:48.528686 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: Python main interpreter initialized at 0x5601b6b729c0 Jul 12 03:34:48.528686 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: python threads support enabled Jul 12 03:34:48.528686 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: your server socket listen backlog is limited to 100 connections Jul 12 03:34:48.528686 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: your mercy for graceful operations on workers is 90 seconds Jul 12 03:34:48.529066 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: mapped 403077 bytes (393 KB) for 2 cores Jul 12 03:34:48.529066 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: *** Operational MODE: preforking *** Jul 12 03:34:48.529414 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: *** uWSGI is running in multiple interpreter mode *** Jul 12 03:34:48.529414 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: spawned uWSGI master process (pid: 86922) Jul 12 03:34:48.529638 dv-cinder-847730-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 12 03:34:48.530203 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: spawned uWSGI worker 1 (pid: 86934, cores: 1) Jul 12 03:34:48.530539 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: spawned uWSGI worker 2 (pid: 86935, cores: 1) Jul 12 03:34:48.530675 dv-cinder-847730-4-iscsi devstack@keystone.service[86922]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 12 03:34:53.878424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 03:34:53.878424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: from cryptography.utils import int_from_bytes Jul 12 03:34:53.879252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 03:34:53.879252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: from cryptography.utils import int_from_bytes Jul 12 03:34:53.879364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 03:34:53.879364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: from cryptography.utils import int_from_bytes Jul 12 03:34:53.879364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 03:34:53.879364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: from cryptography.utils import int_from_bytes Jul 12 03:34:53.927699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 03:34:53.927891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 03:34:53.928051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] command line args: [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 03:34:53.928160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 03:34:53.928281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 03:34:53.928371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 03:34:53.928509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] command line args: [] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 03:34:53.928637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ================================================================================ {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 03:34:53.928714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] admin_token = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.928772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 03:34:53.928948 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ================================================================================ {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 03:34:53.929086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] config_dir = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.929086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.929226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] admin_token = **** {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.929286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] config_source = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.929415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] config_dir = [] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.929493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] debug = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.929563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.929628 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.929898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] config_source = [] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.929898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] debug = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.930017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.930086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.930174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.930313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.930394 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.930493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.930574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.930768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.930894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.930966 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_config_append = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.931025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.931149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.931263 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] list_limit = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.931361 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_dir = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.931455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] log_config_append = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.931551 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_file = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.931692 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.931826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.931994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] log_dir = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.931994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] log_file = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.932269 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.932374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.932517 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.932656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.932744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.932744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.932942 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.933049 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.933126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.933207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.933289 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.933289 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.933515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.933515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.933728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.933848 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.933957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.934068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.934142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.934254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.934371 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.934459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.934574 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.934647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.934762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.934762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.934943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.935026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.935232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.935355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.935355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] publish_errors = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.935636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.935765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.935843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.935959 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] publish_errors = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.936076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.936159 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.936307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.936441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.936522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.936617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.936738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] standard_threads = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.936833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.936988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.937123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.937259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.937336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] standard_threads = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.937413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.937484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.937611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] use_journal = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.937693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.937779 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] use_json = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.937851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.937921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] use_stderr = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.937990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] use_journal = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.938077 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] use_syslog = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.938149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] use_json = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.938228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.938295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] use_stderr = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.938435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.938554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] use_syslog = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.938662 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 03:34:53.938733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.938916 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.939001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.939154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.939235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.939403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.939484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.939654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.939743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.939882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.939959 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.940134 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.940215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.940380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.940462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.940617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.940697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.940866 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.940945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.941089 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.941169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.941334 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.941415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.941567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.941648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.941809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.941897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.942034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.942112 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.942278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.942381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.942521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.942582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.942738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.942820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.942984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.943064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.943216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.943297 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.943465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.943550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.943699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.943820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.943945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.944178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.944377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.944502 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.944597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.944683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.944795 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.944891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.945044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.945168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.945246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.945330 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.945513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.945637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.945727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.945805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.945970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.946110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.946192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.946264 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.946398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.946482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.946636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.946747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.946871 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.946953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.947112 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.947194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.947339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.947431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.947567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.947649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.947800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.947881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.948033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.948206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.948283 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.948353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.948483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.948572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.948705 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.948796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.948933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.949014 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.949163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.949245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.949372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.949461 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.949608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.949714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.949817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.949922 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.950053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.950165 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.950284 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.950387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.950522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.950627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.950773 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.950869 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.950978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.951058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.951261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.951406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.951556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.951684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.951791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.951902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.952044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.952172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.952300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.952415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.952529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.952633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.952784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.952891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.953015 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.953132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.953253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.953381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.953504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.953619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.953731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.953843 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.953965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.954088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.954203 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.954316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.954425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.954539 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.954661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.954767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.954879 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.955003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.955118 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.955232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.955353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.955475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.955589 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.955701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.955814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.955926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.956097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.956217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.956331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.956435 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.956530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.956632 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.956742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.956841 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.956938 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.957033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.957129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.957224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.957330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.957429 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.957534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.957637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.957743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.957841 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.957944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.958043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.958138 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.958234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.958330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.958425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.958533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.958631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.958726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.958824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.958926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.959058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.959250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.959353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.959468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.959567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.959671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.959769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.959873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.959983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.960124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.960240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.960336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.960433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.960534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.960640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.960737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.960869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.960980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.961077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.961730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.961848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.961947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.962059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.962172 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.962284 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.962406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.962529 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.962645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.962759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.962873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.962986 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.963110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.external = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.963110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.963325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.963325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.963543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.963660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] auth.external = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.963781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.963923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.964076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.password = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.964076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] auth.token = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.964296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.964414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.964528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.964643 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.964765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] auth.password = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.964880 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.965000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] auth.token = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.965115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.965237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.965352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.965474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.965597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.965713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.965826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.965940 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.966053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.966175 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.966295 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.966411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.966537 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.966654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.966865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.966994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] credential.caching = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.967111 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.967226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.967339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.967453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.967568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.967690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.967807 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.967929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.968062 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.968178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.968292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.968422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.968538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.968653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.968766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.968888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.969003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.969125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.969241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.969356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.969469 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.969583 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.969697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.969819 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.969938 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.970051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.970165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.970285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.970400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.970522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.970638 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.970751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] federation.caching = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.970864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.970979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.971092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.971215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.971331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.971445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.971574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.971689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.971803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.971925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.972055 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.972171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.972285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.972399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.972525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.972641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.972755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.972868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.972983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.973096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.973293 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.973410 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.973524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.973637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.caching = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.973750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.973864 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.973992 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.974110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.974224 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.974337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.974453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.974566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.974688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.974803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.974917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.975030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.975144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.975258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.975380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.975380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.975596 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.975710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.975823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.975936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.976113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.976113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.976333 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.976447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.976561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.976674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.976800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.976915 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.977029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.977142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.977255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.977369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.977491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.977606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.977720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.977720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.977932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.978045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.978167 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.978282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.978396 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.978508 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.978622 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.978740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.978863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.978978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.979091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.979204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.979318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.979452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.979576 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.979691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.979804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.979917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.980044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.980171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.980296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.980412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.980525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.980639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.980752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.980865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.980987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.981102 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.981215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.981328 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.981443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.981555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.981677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.981791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.981904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.982017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.982131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.982244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.982391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.982534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.982659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.982774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.982896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.983011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.983133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.983248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.983362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.983477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.983590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.983704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.983826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.983941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.984126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.984246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.984365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.984488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.984605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.984726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.984843 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.984962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.985076 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.985200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.985316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.985429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.985543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.985657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.985771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.985892 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.986008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.986121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.986235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.986348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.986461 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.986583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.986699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.986812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.986926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.987045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.987160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.987282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.987398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.987511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.987631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.987746 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.987860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.987982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.988115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.988229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.988343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.988456 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.988569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.988695 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.988812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.988926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.989039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.989152 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.989265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.989386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.989501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.989615 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.989615 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.989829 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.989942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.990065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.990179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.990292 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.990405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.990519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.990632 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.990754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.990754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.990970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.990970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.991182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.991302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.991424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.991539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.991653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.991766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.991880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.991993 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.992143 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.992260 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.992373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.992486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.992599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.992712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.992834 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.992950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.993064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.993177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.993292 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.993406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.993527 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.993642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.993755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.993868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.993982 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.994095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.994217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.994335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.994493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] resource.caching = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.994606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.994720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.994833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.994955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.995073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.995186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.995300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.995412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.995526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] role.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.995716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.995833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] role.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.995946 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.996095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.996210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.996323 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.996449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.996565 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.996686 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.996800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.996913 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] role.caching = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.997035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.997152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] role.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.997272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.997386 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.997499 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.997612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.997733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.997839 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.997951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.998110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.998228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.998341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.998464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.998580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.998695 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.998809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.998921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.999035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.999157 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.999273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.999273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.999486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.999599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.999712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.999837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:53.999953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.000093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.000207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.000322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.000435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.000435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.000666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.000780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.000893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.001006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.001119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.001241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.001357 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.001471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.001586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.001699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.001812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.001936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.002087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.002204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.002317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.002430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.002554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.002669 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.002781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.002897 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.003010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.003122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.003243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.003358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.003472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.003584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.003697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.003810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.003931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.004085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.004199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.004315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.004315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.004529 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.004529 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.004757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.004871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.004984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.005097 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] token.caching = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.005210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.005331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.005446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.005559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.005673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.005785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.005898 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.005898 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.006121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.006236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.006349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.006462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.006575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.006696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.006811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.006923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.007036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.007149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.007261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.007381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.007496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.007496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.007709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.007822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.007934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.connection = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.008076 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.008205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.008323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.008443 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.008557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.008672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.008784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.008905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.008905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.009119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.009233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.009345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.connection = **** {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.009468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.009592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.009708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.009830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.009945 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.010057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.010170 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.010283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.010395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.010517 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.010631 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.010745 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.010858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.010972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.011085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.011207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.011322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.011435 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.011548 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.011661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.011774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.011895 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.012073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.012189 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.012302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86934) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 03:34:54.012426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.012426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.012426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.012426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.012426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.012426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 03:34:54.012426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86935) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 03:34:54.041101 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.041326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.041548 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.041642 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.041852 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.042079 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.050064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.050571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.077707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.077913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.078043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.078296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.221496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.221714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.269573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.269800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.270305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.270553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.332381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.332582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.332813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.332964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.333044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.333107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.333281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.333411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.333554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.333698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.333698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.333937 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.357747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.357948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.358098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.358182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.358244 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.358390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.358515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.358613 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.358681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.358860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.358990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.359129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.359269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.359269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.369950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.370411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.382568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.382777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.382924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.383007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.383136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.383218 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.383364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.383461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.383581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.383802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.383895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86934) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.384032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.384283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86935) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 03:34:54.403477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.403633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.403775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 03:34:54.403857 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 03:34:54.404093 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.404308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.404555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 03:34:54.404694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 03:34:54.404776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.404859 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.404936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.405214 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.405340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.405478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.405557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.405714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.416718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.416992 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.417250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.417376 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.417534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.417619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.417783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.418059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.425721 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.426325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.428965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.429240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.429511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.429636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.429790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.429979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.430071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.430218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.430299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.430425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.430515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.430651 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.430776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.430936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.431060 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.431184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.431323 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.431453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.431580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.431709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.431915 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.432187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.457476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.458238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.459042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.459313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.459840 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.460142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.464508 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.464776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.465167 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.465292 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.465604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.465993 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.472403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.472630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.473251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.473531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.477050 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.477331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.477602 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.477872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.478015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.478096 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.478287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.478417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.478558 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.478698 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.478827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.478968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.479050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.479275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.479539 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.479771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.497190 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.497454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.498250 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.498536 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.502045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.503142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.505666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.505954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.506750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.507040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.510985 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.512151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.514660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.515797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.518342 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.518633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.518918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.519211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.519406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.519524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.519674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.519821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.520054 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.520301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.520573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.520861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.538448 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.538723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.538979 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.539213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.539636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.539914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.540201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.540463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.547405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.548665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.549114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.549380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.550334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.550611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.552511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.552783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.553059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.553334 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.553626 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.553934 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.554210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.554489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.554768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.555028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.566710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.566981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.567256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.567540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.567824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.568154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.568430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.568715 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.568995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.569281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.569406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.569665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.570823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.571137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.587334 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.588944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.590581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.592224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.593870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.595509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.595637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.597243 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.597370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.597489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.597815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.598893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.599168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.599449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.605121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.606742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.606871 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.608446 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.610103 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.610384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.610768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.611048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.611626 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.611925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.612347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.612625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.620038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.620345 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.620615 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.621716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.622006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.622281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.628762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86934) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.628762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.629068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.629338 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.629599 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.629971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.630178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86935) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 03:34:54.630319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.630459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.630540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.630710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.630836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.630975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.631070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.631229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.631355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.631550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.631730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.631818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.631893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.632069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86934) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.632152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.632440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.632722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.632977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.633234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.633492 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.633767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86935) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 03:34:54.669801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5601b6b729c0 pid: 86934 (default app) Jul 12 03:34:54.671351 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5601b6b729c0 pid: 86935 (default app) Jul 12 03:34:54.673324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec69fb-2d4f-44c7-8dd5-379498fe87ac None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:34:54.673614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec69fb-2d4f-44c7-8dd5-379498fe87ac None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:34:54.673893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec69fb-2d4f-44c7-8dd5-379498fe87ac None None] PATH_INFO: `/v3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:34:54.678220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Tue Jul 12 03:34:54 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:02.368652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b5f93-ffa6-4c49-b02f-473a16a82b34 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:02.369541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b5f93-ffa6-4c49-b02f-473a16a82b34 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:02.369541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b5f93-ffa6-4c49-b02f-473a16a82b34 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:02.376147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:02 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:02.381650 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fdf84-c76b-4b53-9aee-1928cf55f5a0 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:02.382105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fdf84-c76b-4b53-9aee-1928cf55f5a0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:02.382196 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fdf84-c76b-4b53-9aee-1928cf55f5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:02.458292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: INFO dbcounter [None req-cb7fdf84-c76b-4b53-9aee-1928cf55f5a0 None None] Registered counter for database keystone Jul 12 03:35:02.483367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG oslo_db.sqlalchemy.engines [None req-cb7fdf84-c76b-4b53-9aee-1928cf55f5a0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86934) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 03:35:02.495554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writer thread running {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 03:35:02.679329 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG passlib.handlers.bcrypt [None req-cb7fdf84-c76b-4b53-9aee-1928cf55f5a0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86934) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 03:35:02.679650 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG passlib.handlers.bcrypt [None req-cb7fdf84-c76b-4b53-9aee-1928cf55f5a0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86934) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 03:35:02.735114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-cb7fdf84-c76b-4b53-9aee-1928cf55f5a0 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:02.736802 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cb7fdf84-c76b-4b53-9aee-1928cf55f5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:02.868853 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 488 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 03:35:03.065285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b5f93-ffa6-4c49-b02f-473a16a82b34 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:03.135053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: INFO dbcounter [None req-e83b5f93-ffa6-4c49-b02f-473a16a82b34 None None] Registered counter for database keystone Jul 12 03:35:03.157046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG oslo_db.sqlalchemy.engines [None req-e83b5f93-ffa6-4c49-b02f-473a16a82b34 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86935) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 03:35:03.165952 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writer thread running {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 03:35:03.289861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b5f93-ffa6-4c49-b02f-473a16a82b34 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:03.294548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37329d-2f40-4d86-998a-fdf4a8cbc4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:03.295212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37329d-2f40-4d86-998a-fdf4a8cbc4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:03.295500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37329d-2f40-4d86-998a-fdf4a8cbc4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:03.295778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37329d-2f40-4d86-998a-fdf4a8cbc4f0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:03.304868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37329d-2f40-4d86-998a-fdf4a8cbc4f0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:03.355804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3c37329d-2f40-4d86-998a-fdf4a8cbc4f0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 03:35:03.356169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37329d-2f40-4d86-998a-fdf4a8cbc4f0 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:03.366400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Jul 12 03:35:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 03:35:03.373562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fdf84-c76b-4b53-9aee-1928cf55f5a0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:03.395728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fdf84-c76b-4b53-9aee-1928cf55f5a0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:03.404028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc75f39-8e1b-48d1-b86b-e7b7d053ed28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:03.405724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc75f39-8e1b-48d1-b86b-e7b7d053ed28 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:03.406182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc75f39-8e1b-48d1-b86b-e7b7d053ed28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:03.406590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc75f39-8e1b-48d1-b86b-e7b7d053ed28 admin admin] PATH_INFO: `/v3/projects/579b7a08aae94d3fa59617c603862f30` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:03.413446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc75f39-8e1b-48d1-b86b-e7b7d053ed28 admin admin] RBAC: Authorizing `identity:get_project(project_id=579b7a08aae94d3fa59617c603862f30)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:03.466549 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2dc75f39-8e1b-48d1-b86b-e7b7d053ed28 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 03:35:03.466889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc75f39-8e1b-48d1-b86b-e7b7d053ed28 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:03.471573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Tue Jul 12 03:35:03 2022] GET /identity/v3/projects/579b7a08aae94d3fa59617c603862f30 => generated 344 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:05.476056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff8a66-3688-4e8b-8974-871c06d8637d None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:05.476652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff8a66-3688-4e8b-8974-871c06d8637d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:05.476652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff8a66-3688-4e8b-8974-871c06d8637d None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:05.480812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:35:05.487704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48daa227-037d-477f-9235-ae88ed3ff50e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:05.488485 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48daa227-037d-477f-9235-ae88ed3ff50e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:05.488630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48daa227-037d-477f-9235-ae88ed3ff50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:05.514603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-48daa227-037d-477f-9235-ae88ed3ff50e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:05.516185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-48daa227-037d-477f-9235-ae88ed3ff50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:05.527413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 03:35:05.562525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff8a66-3688-4e8b-8974-871c06d8637d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:05.585008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff8a66-3688-4e8b-8974-871c06d8637d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:05.594212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a0fb7-59a7-418b-89ac-918e0544e704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:05.594881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a0fb7-59a7-418b-89ac-918e0544e704 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:05.595163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a0fb7-59a7-418b-89ac-918e0544e704 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:05.595457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a0fb7-59a7-418b-89ac-918e0544e704 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:05.604182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a0fb7-59a7-418b-89ac-918e0544e704 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:05.605986 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a0fb7-59a7-418b-89ac-918e0544e704 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:05.614860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Tue Jul 12 03:35:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:35:05.620429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48daa227-037d-477f-9235-ae88ed3ff50e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:05.630534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48daa227-037d-477f-9235-ae88ed3ff50e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:05.636400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9f17b-78bf-4088-893f-dd3f190e88e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:05.637276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9f17b-78bf-4088-893f-dd3f190e88e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:05.637838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9f17b-78bf-4088-893f-dd3f190e88e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:05.638129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9f17b-78bf-4088-893f-dd3f190e88e2 admin admin] PATH_INFO: `/v3/users/819f4355d22549de99860edd3df36888` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:05.639699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9f17b-78bf-4088-893f-dd3f190e88e2 admin admin] RBAC: Authorizing `identity:get_user(user_id=819f4355d22549de99860edd3df36888)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:05.641674 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9f17b-78bf-4088-893f-dd3f190e88e2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:05.643285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Tue Jul 12 03:35:05 2022] GET /identity/v3/users/819f4355d22549de99860edd3df36888 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:07.698291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6d5e0-1ec8-4321-9295-01479c3c4a0d None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:07.698835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6d5e0-1ec8-4321-9295-01479c3c4a0d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:07.698835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6d5e0-1ec8-4321-9295-01479c3c4a0d None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:07.700264 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:07.705561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c038e-021e-42c8-bcd8-fd52b7f748d4 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:07.705971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c038e-021e-42c8-bcd8-fd52b7f748d4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:07.706142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c038e-021e-42c8-bcd8-fd52b7f748d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:07.723617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5a8c038e-021e-42c8-bcd8-fd52b7f748d4 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:07.724851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5a8c038e-021e-42c8-bcd8-fd52b7f748d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:07.738323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0789a-7211-4a52-be7d-529d7417d669 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:07.738610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0789a-7211-4a52-be7d-529d7417d669 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:07.738886 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e0789a-7211-4a52-be7d-529d7417d669 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:07.740231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:35:07.745022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aa15c-4f6c-441b-af9d-f0e01175ed6f None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:07.745316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aa15c-4f6c-441b-af9d-f0e01175ed6f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:07.745598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aa15c-4f6c-441b-af9d-f0e01175ed6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:07.785750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG passlib.handlers.bcrypt [None req-e54aa15c-4f6c-441b-af9d-f0e01175ed6f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86935) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 03:35:07.786251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG passlib.handlers.bcrypt [None req-e54aa15c-4f6c-441b-af9d-f0e01175ed6f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86935) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 03:35:07.812370 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:07.831616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-e54aa15c-4f6c-441b-af9d-f0e01175ed6f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:07.833294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e54aa15c-4f6c-441b-af9d-f0e01175ed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:07.835815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c038e-021e-42c8-bcd8-fd52b7f748d4 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:07.860772 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:07.933054 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:07.944949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c038e-021e-42c8-bcd8-fd52b7f748d4 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:07.955201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54aa15c-4f6c-441b-af9d-f0e01175ed6f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:07.981797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eced0c-76dd-4704-884a-b017f2e7e013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:07.982623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eced0c-76dd-4704-884a-b017f2e7e013 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:07.982911 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eced0c-76dd-4704-884a-b017f2e7e013 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:07.983194 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eced0c-76dd-4704-884a-b017f2e7e013 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:07.995485 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eced0c-76dd-4704-884a-b017f2e7e013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-54eced0c-76dd-4704-884a-b017f2e7e013 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:08.006965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:08.007867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:08.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:35:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:08.014193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eced0c-76dd-4704-884a-b017f2e7e013 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:08.056489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54aa15c-4f6c-441b-af9d-f0e01175ed6f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:08.058796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:08.094395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cae4c0-3099-4bca-9c78-b97d29ca6187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:08.095294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cae4c0-3099-4bca-9c78-b97d29ca6187 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:08.095588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cae4c0-3099-4bca-9c78-b97d29ca6187 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:08.095760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:08.095871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cae4c0-3099-4bca-9c78-b97d29ca6187 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:08.114508 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eced0c-76dd-4704-884a-b017f2e7e013 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:08.149603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23406ca4-06fb-4b06-b6bb-c620affad742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:08.150274 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23406ca4-06fb-4b06-b6bb-c620affad742 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:08.150563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23406ca4-06fb-4b06-b6bb-c620affad742 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:08.150845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23406ca4-06fb-4b06-b6bb-c620affad742 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:08.151600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23406ca4-06fb-4b06-b6bb-c620affad742 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:08.151909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23406ca4-06fb-4b06-b6bb-c620affad742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:08.153364 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23406ca4-06fb-4b06-b6bb-c620affad742 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:08.163543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cae4c0-3099-4bca-9c78-b97d29ca6187 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:08.165100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cae4c0-3099-4bca-9c78-b97d29ca6187 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:08.165683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:08.171600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23406ca4-06fb-4b06-b6bb-c620affad742 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:08.262523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 03:35:08.268091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cae4c0-3099-4bca-9c78-b97d29ca6187 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:08.268716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23406ca4-06fb-4b06-b6bb-c620affad742 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:08.270776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:08.303252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92733c14-bca5-4d2a-9358-b11886d2039c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:08.304159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92733c14-bca5-4d2a-9358-b11886d2039c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:08.304441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92733c14-bca5-4d2a-9358-b11886d2039c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:08.304716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92733c14-bca5-4d2a-9358-b11886d2039c None admin] PATH_INFO: `/v3/users/819f4355d22549de99860edd3df36888` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:08.305910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92733c14-bca5-4d2a-9358-b11886d2039c None admin] RBAC: Authorizing `identity:get_user(user_id=819f4355d22549de99860edd3df36888)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:08.307396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92733c14-bca5-4d2a-9358-b11886d2039c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:08.309038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:08 2022] GET /identity/v3/users/819f4355d22549de99860edd3df36888 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:08.314432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92733c14-bca5-4d2a-9358-b11886d2039c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:08.328497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:08.368025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cae4c0-3099-4bca-9c78-b97d29ca6187 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:08.405301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4097d4c-90e0-4588-b079-8e2c60d7107d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:08.406125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4097d4c-90e0-4588-b079-8e2c60d7107d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:08.406406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4097d4c-90e0-4588-b079-8e2c60d7107d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:08.406684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4097d4c-90e0-4588-b079-8e2c60d7107d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:08.414795 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92733c14-bca5-4d2a-9358-b11886d2039c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:08.425330 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4097d4c-90e0-4588-b079-8e2c60d7107d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:08.426805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4097d4c-90e0-4588-b079-8e2c60d7107d None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-d4097d4c-90e0-4588-b079-8e2c60d7107d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:35:08.447030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:08.447848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:08.448672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:35:08.448672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:35:08.448672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:35:08.448672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:08.448672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:35:08.451143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae3899-0758-4efa-93f5-dcb25ae4c1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:08.451942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae3899-0758-4efa-93f5-dcb25ae4c1fc None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:08.452257 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae3899-0758-4efa-93f5-dcb25ae4c1fc None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:08.452533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae3899-0758-4efa-93f5-dcb25ae4c1fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:08.453759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae3899-0758-4efa-93f5-dcb25ae4c1fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:08.454206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4097d4c-90e0-4588-b079-8e2c60d7107d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:08.455254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ae3899-0758-4efa-93f5-dcb25ae4c1fc None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:08.457009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:08.462515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae3899-0758-4efa-93f5-dcb25ae4c1fc None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:08.510519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:08.552677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:08.557348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4097d4c-90e0-4588-b079-8e2c60d7107d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:08.571609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae3899-0758-4efa-93f5-dcb25ae4c1fc None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:08.590463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f606630-5063-4b45-9bee-1cfa383cde00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:08.591143 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f606630-5063-4b45-9bee-1cfa383cde00 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:08.591417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f606630-5063-4b45-9bee-1cfa383cde00 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:08.591696 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f606630-5063-4b45-9bee-1cfa383cde00 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:08.592449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f606630-5063-4b45-9bee-1cfa383cde00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:08.592754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f606630-5063-4b45-9bee-1cfa383cde00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:08.594167 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f606630-5063-4b45-9bee-1cfa383cde00 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:08.604591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c763099-3f45-442b-b8e2-50782be99d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:08.605241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c763099-3f45-442b-b8e2-50782be99d06 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:08.605529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c763099-3f45-442b-b8e2-50782be99d06 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:08.605809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c763099-3f45-442b-b8e2-50782be99d06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:08.606648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c763099-3f45-442b-b8e2-50782be99d06 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.domain.id=default, user.id=819f4355d22549de99860edd3df36888) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:08.606967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c763099-3f45-442b-b8e2-50782be99d06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:08.608440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c763099-3f45-442b-b8e2-50782be99d06 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:08.611547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:08.615549 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Jul 12 03:35:08 2022] GET /identity/v3/role_assignments?user.id=819f4355d22549de99860edd3df36888&scope.domain.id=default&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:11.042114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbfe7f-494a-4e66-af1e-32683602b42f None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:11.042629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbfe7f-494a-4e66-af1e-32683602b42f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:11.042731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbfe7f-494a-4e66-af1e-32683602b42f None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:11.044359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:11.046389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e072d8-8e75-493c-a189-28de239edc2b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:11.046883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e072d8-8e75-493c-a189-28de239edc2b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:11.047172 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e072d8-8e75-493c-a189-28de239edc2b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:11.048943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 12/23] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:11.053897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff829af-d807-41de-98b9-84d8e2cdecc9 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:11.054190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff829af-d807-41de-98b9-84d8e2cdecc9 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:11.054467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff829af-d807-41de-98b9-84d8e2cdecc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:11.057500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853413c-49aa-4d64-adfb-1b954c22f4aa None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:11.057788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853413c-49aa-4d64-adfb-1b954c22f4aa None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:11.058061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853413c-49aa-4d64-adfb-1b954c22f4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:11.085572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-f853413c-49aa-4d64-adfb-1b954c22f4aa None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:11.086758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f853413c-49aa-4d64-adfb-1b954c22f4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:11.092586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-7ff829af-d807-41de-98b9-84d8e2cdecc9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:11.093742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7ff829af-d807-41de-98b9-84d8e2cdecc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:11.195236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 12/24] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:11.196661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd109c18-82d0-472b-ab3c-5ef3576cb352 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:11.196972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd109c18-82d0-472b-ab3c-5ef3576cb352 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:11.197251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd109c18-82d0-472b-ab3c-5ef3576cb352 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:11.198686 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 13/25] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:11.203967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5349f35-a7ce-48c0-a532-c0a0bde665bf None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:11.204285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5349f35-a7ce-48c0-a532-c0a0bde665bf None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:11.204620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5349f35-a7ce-48c0-a532-c0a0bde665bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:11.209259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 13/26] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:11.224124 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d5349f35-a7ce-48c0-a532-c0a0bde665bf None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:11.225235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d5349f35-a7ce-48c0-a532-c0a0bde665bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:11.241359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853413c-49aa-4d64-adfb-1b954c22f4aa None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:11.305736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:11.328644 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:11.352505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853413c-49aa-4d64-adfb-1b954c22f4aa None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:11.370042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 14/27] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:11.371204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5349f35-a7ce-48c0-a532-c0a0bde665bf None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:11.420653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d6e26-87fd-4b1e-99e9-e3157f83a2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:11.422401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d6e26-87fd-4b1e-99e9-e3157f83a2ac None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:11.423051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d6e26-87fd-4b1e-99e9-e3157f83a2ac None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:11.424621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d6e26-87fd-4b1e-99e9-e3157f83a2ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:11.446728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d6e26-87fd-4b1e-99e9-e3157f83a2ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-db7d6e26-87fd-4b1e-99e9-e3157f83a2ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:11.454737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:11.455546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:11.456303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:11.456303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:11.456303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:11.456303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:11.456303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:11.456303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:11.456303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:11.456303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:11.456303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:11.456303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:11.456303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:11.456303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:11.458976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 14/28] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:35:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:11.460057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d6e26-87fd-4b1e-99e9-e3157f83a2ac None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:11.599767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5349f35-a7ce-48c0-a532-c0a0bde665bf None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:11.636936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d6e26-87fd-4b1e-99e9-e3157f83a2ac None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:11.666995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b8f12-3162-4e3a-8ead-92bb8a0746a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:11.667806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b8f12-3162-4e3a-8ead-92bb8a0746a8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:11.668252 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b8f12-3162-4e3a-8ead-92bb8a0746a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:11.668774 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b8f12-3162-4e3a-8ead-92bb8a0746a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:11.693174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b8f12-3162-4e3a-8ead-92bb8a0746a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:11.695075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b8f12-3162-4e3a-8ead-92bb8a0746a8 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:11.708095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d67ac-60ec-4381-8234-90a8700237eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:11.708978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d67ac-60ec-4381-8234-90a8700237eb None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:11.709394 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d67ac-60ec-4381-8234-90a8700237eb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:11.709798 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d67ac-60ec-4381-8234-90a8700237eb None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:11.710673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d67ac-60ec-4381-8234-90a8700237eb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:11.717547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d67ac-60ec-4381-8234-90a8700237eb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-134b8f12-3162-4e3a-8ead-92bb8a0746a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:35:11.721879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:11.722828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:11.723789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:35:11.723789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:35:11.723789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:35:11.723789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:11.726047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 15/29] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:11.726300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:11.727272 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b8f12-3162-4e3a-8ead-92bb8a0746a8 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:11.768347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:11.877822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 15/30] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Jul 12 03:35:11 2022] POST /identity/v3/roles => generated 219 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:11.878829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d67ac-60ec-4381-8234-90a8700237eb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:11.930476 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b8f12-3162-4e3a-8ead-92bb8a0746a8 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:11.985407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da1a3c-aac1-4271-8dd2-e61fab2f075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:11.986594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da1a3c-aac1-4271-8dd2-e61fab2f075b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:11.987022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da1a3c-aac1-4271-8dd2-e61fab2f075b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:11.987441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da1a3c-aac1-4271-8dd2-e61fab2f075b None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:11.988465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da1a3c-aac1-4271-8dd2-e61fab2f075b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:11.988927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da1a3c-aac1-4271-8dd2-e61fab2f075b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:11.990516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da1a3c-aac1-4271-8dd2-e61fab2f075b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:11.994823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d67ac-60ec-4381-8234-90a8700237eb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:12.005230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 16/31] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:12.011441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f753bf7-ed18-4c8c-b6cc-d40c057cda86 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.011701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f753bf7-ed18-4c8c-b6cc-d40c057cda86 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.011981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f753bf7-ed18-4c8c-b6cc-d40c057cda86 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.014016 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 17/32] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:12.015174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56e908-5d93-4ad5-9eac-a875f42c920c None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.015464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56e908-5d93-4ad5-9eac-a875f42c920c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.015740 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56e908-5d93-4ad5-9eac-a875f42c920c None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.021106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 18/33] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:12.021997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56e908-5d93-4ad5-9eac-a875f42c920c None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:12.035681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb5e8f-ff3a-48ca-b94f-dceedfa4406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:12.036730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb5e8f-ff3a-48ca-b94f-dceedfa4406a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.037186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb5e8f-ff3a-48ca-b94f-dceedfa4406a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.037624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb5e8f-ff3a-48ca-b94f-dceedfa4406a None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.038585 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb5e8f-ff3a-48ca-b94f-dceedfa4406a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:12.039108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb5e8f-ff3a-48ca-b94f-dceedfa4406a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:12.040959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb5e8f-ff3a-48ca-b94f-dceedfa4406a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:12.061712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 16/34] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:12.064265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ea413-f817-453c-8478-43002c7cfd64 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.064766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ea413-f817-453c-8478-43002c7cfd64 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.065208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ea413-f817-453c-8478-43002c7cfd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.089531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-fa9ea413-f817-453c-8478-43002c7cfd64 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:12.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fa9ea413-f817-453c-8478-43002c7cfd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:12.150087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:12.163467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:12.166535 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56e908-5d93-4ad5-9eac-a875f42c920c None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:12.184193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 17/35] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:12.185565 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a9a06-086c-4c10-b2ad-4416b5864374 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.185864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a9a06-086c-4c10-b2ad-4416b5864374 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.186142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a9a06-086c-4c10-b2ad-4416b5864374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.207451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-be3a9a06-086c-4c10-b2ad-4416b5864374 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:12.208987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-be3a9a06-086c-4c10-b2ad-4416b5864374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:12.211864 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd978d-e7cf-4e92-92e1-c76e80671010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:12.213016 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd978d-e7cf-4e92-92e1-c76e80671010 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.213314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd978d-e7cf-4e92-92e1-c76e80671010 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.213592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd978d-e7cf-4e92-92e1-c76e80671010 None admin] PATH_INFO: `/v3/users/819f4355d22549de99860edd3df36888` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.215176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd978d-e7cf-4e92-92e1-c76e80671010 None admin] RBAC: Authorizing `identity:get_user(user_id=819f4355d22549de99860edd3df36888)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:12.216775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd978d-e7cf-4e92-92e1-c76e80671010 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:12.222931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 19/36] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:12 2022] GET /identity/v3/users/819f4355d22549de99860edd3df36888 => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:12.223836 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd978d-e7cf-4e92-92e1-c76e80671010 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:12.292365 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 18/37] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:12.294702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a9a06-086c-4c10-b2ad-4416b5864374 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:12.329335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd978d-e7cf-4e92-92e1-c76e80671010 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:12.364659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a8ea9-4f8d-4705-a232-db53f81624c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:12.365578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a8ea9-4f8d-4705-a232-db53f81624c1 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.366005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a8ea9-4f8d-4705-a232-db53f81624c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.366423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a8ea9-4f8d-4705-a232-db53f81624c1 None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.367315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a8ea9-4f8d-4705-a232-db53f81624c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:12.368845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a8ea9-4f8d-4705-a232-db53f81624c1 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:12.405122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a9a06-086c-4c10-b2ad-4416b5864374 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:12.423347 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:12.448423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398e116-547d-439f-a60d-ebbbac5f82f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:12.449430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398e116-547d-439f-a60d-ebbbac5f82f8 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.449873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398e116-547d-439f-a60d-ebbbac5f82f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.450298 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398e116-547d-439f-a60d-ebbbac5f82f8 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.451250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398e116-547d-439f-a60d-ebbbac5f82f8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:12.452829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398e116-547d-439f-a60d-ebbbac5f82f8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:12.467744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 19/38] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Jul 12 03:35:12 2022] POST /identity/v3/roles => generated 223 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:12.468892 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398e116-547d-439f-a60d-ebbbac5f82f8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:12.504066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 20/39] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:35:12 2022] POST /identity/v3/projects => generated 301 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:12.505068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a8ea9-4f8d-4705-a232-db53f81624c1 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:12.528575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:12.664409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398e116-547d-439f-a60d-ebbbac5f82f8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:12.690381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a8ea9-4f8d-4705-a232-db53f81624c1 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:12.716857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc6f9c-3eda-4f6f-acae-e4f1e7694cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:12.717820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc6f9c-3eda-4f6f-acae-e4f1e7694cfa None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.718108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc6f9c-3eda-4f6f-acae-e4f1e7694cfa None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.718382 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc6f9c-3eda-4f6f-acae-e4f1e7694cfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.719939 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc6f9c-3eda-4f6f-acae-e4f1e7694cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:12.721561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc6f9c-3eda-4f6f-acae-e4f1e7694cfa None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:12.723270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 20/40] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:12.724624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3066f-4329-4263-b28a-6d356cdac647 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.724915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3066f-4329-4263-b28a-6d356cdac647 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.725229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3066f-4329-4263-b28a-6d356cdac647 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.726601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 21/41] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:12.732395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368f492-31ba-49bd-824b-0a1ea9a4dfb3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.732723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368f492-31ba-49bd-824b-0a1ea9a4dfb3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.732964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368f492-31ba-49bd-824b-0a1ea9a4dfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.736416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b8b7b-3500-4295-a608-c612b9a89add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:12.737079 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b8b7b-3500-4295-a608-c612b9a89add None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.737368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b8b7b-3500-4295-a608-c612b9a89add None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.737646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b8b7b-3500-4295-a608-c612b9a89add None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.738409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b8b7b-3500-4295-a608-c612b9a89add None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:12.741197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b8b7b-3500-4295-a608-c612b9a89add None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:12.759805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 21/42] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Jul 12 03:35:12 2022] POST /identity/v3/roles => generated 225 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:12.760516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b8b7b-3500-4295-a608-c612b9a89add None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:12.854024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:12.871006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9368f492-31ba-49bd-824b-0a1ea9a4dfb3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:12.872311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9368f492-31ba-49bd-824b-0a1ea9a4dfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:12.892243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b8b7b-3500-4295-a608-c612b9a89add None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:12.912428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:12.929942 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4931608-d909-4ede-b3a3-1b28bd79cb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:12.931774 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4931608-d909-4ede-b3a3-1b28bd79cb44 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:12.931774 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4931608-d909-4ede-b3a3-1b28bd79cb44 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:12.931774 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4931608-d909-4ede-b3a3-1b28bd79cb44 None admin] PATH_INFO: `/v3/domains/default/users/819f4355d22549de99860edd3df36888/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:12.934708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4931608-d909-4ede-b3a3-1b28bd79cb44 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=819f4355d22549de99860edd3df36888, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:12.936223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4931608-d909-4ede-b3a3-1b28bd79cb44 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:12.959631 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 22/43] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:12.959967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 22/44] 10.222.0.20 () {58 vars in 1361 bytes} [Tue Jul 12 03:35:12 2022] PUT /identity/v3/domains/default/users/819f4355d22549de99860edd3df36888/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:35:12.990103 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368f492-31ba-49bd-824b-0a1ea9a4dfb3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:13.100844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368f492-31ba-49bd-824b-0a1ea9a4dfb3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:13.141151 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5147d1-491e-414e-b0ec-c2ea5d18b557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:13.142112 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5147d1-491e-414e-b0ec-c2ea5d18b557 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:13.142485 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5147d1-491e-414e-b0ec-c2ea5d18b557 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:13.142766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5147d1-491e-414e-b0ec-c2ea5d18b557 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:13.144159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5147d1-491e-414e-b0ec-c2ea5d18b557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:13.145682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5147d1-491e-414e-b0ec-c2ea5d18b557 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:13.147362 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 23/45] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:13.153437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4931608-d909-4ede-b3a3-1b28bd79cb44 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:13.263943 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4931608-d909-4ede-b3a3-1b28bd79cb44 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:13.407443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:13.414845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff61d52-a05f-4b14-88af-5a3984317bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:13.415551 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff61d52-a05f-4b14-88af-5a3984317bc5 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:13.415882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff61d52-a05f-4b14-88af-5a3984317bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:13.416327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff61d52-a05f-4b14-88af-5a3984317bc5 None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:13.417168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff61d52-a05f-4b14-88af-5a3984317bc5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:13.419122 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff61d52-a05f-4b14-88af-5a3984317bc5 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:13.446385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 23/46] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:35:13 2022] POST /identity/v3/projects => generated 312 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:14.971729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561465ae-b365-4917-96d9-b3364a3e0278 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:14.972222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561465ae-b365-4917-96d9-b3364a3e0278 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:14.972333 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561465ae-b365-4917-96d9-b3364a3e0278 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:14.973707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 24/47] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:14.979115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d2d11-5837-4bde-a6dd-b96d7e8d9fa9 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:14.979525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d2d11-5837-4bde-a6dd-b96d7e8d9fa9 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:14.979714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d2d11-5837-4bde-a6dd-b96d7e8d9fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:14.998067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a60d2d11-5837-4bde-a6dd-b96d7e8d9fa9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:14.999612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a60d2d11-5837-4bde-a6dd-b96d7e8d9fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:15.070504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 24/48] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:15.093660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561465ae-b365-4917-96d9-b3364a3e0278 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:15.138136 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:15.198905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561465ae-b365-4917-96d9-b3364a3e0278 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:15.233588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba45f1b-2729-4620-bad3-c2457fe6ba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:15.234317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba45f1b-2729-4620-bad3-c2457fe6ba23 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:15.234602 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba45f1b-2729-4620-bad3-c2457fe6ba23 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:15.234877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba45f1b-2729-4620-bad3-c2457fe6ba23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:15.245614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba45f1b-2729-4620-bad3-c2457fe6ba23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-bba45f1b-2729-4620-bad3-c2457fe6ba23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:15.257675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:15.258547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:15.259384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 25/49] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:35:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:15.265244 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d2d11-5837-4bde-a6dd-b96d7e8d9fa9 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:15.366644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d2d11-5837-4bde-a6dd-b96d7e8d9fa9 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:15.379882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:15.404224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad1634d-a9e1-480f-b447-245ce83ca52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:15.404897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1634d-a9e1-480f-b447-245ce83ca52c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:15.405184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1634d-a9e1-480f-b447-245ce83ca52c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:15.405459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1634d-a9e1-480f-b447-245ce83ca52c None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:15.406223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1634d-a9e1-480f-b447-245ce83ca52c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:15.406535 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1634d-a9e1-480f-b447-245ce83ca52c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:15.407957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad1634d-a9e1-480f-b447-245ce83ca52c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:15.416437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 25/50] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:15.422631 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba45f1b-2729-4620-bad3-c2457fe6ba23 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:15.468468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bfb40-879c-45f3-8e54-fef2a1a74a28 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:15.468761 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bfb40-879c-45f3-8e54-fef2a1a74a28 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:15.469096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bfb40-879c-45f3-8e54-fef2a1a74a28 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:15.470765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 26/51] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:35:15.476194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4111940-9235-478b-a868-86ddc1b1582b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:15.476488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4111940-9235-478b-a868-86ddc1b1582b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:15.476774 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4111940-9235-478b-a868-86ddc1b1582b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:15.494218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d4111940-9235-478b-a868-86ddc1b1582b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:15.495313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d4111940-9235-478b-a868-86ddc1b1582b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:15.521235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:15.524125 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba45f1b-2729-4620-bad3-c2457fe6ba23 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:15.559851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e558c-52e6-43f0-a456-7b172fb81870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:15.560776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e558c-52e6-43f0-a456-7b172fb81870 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:15.561064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e558c-52e6-43f0-a456-7b172fb81870 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:15.561343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e558c-52e6-43f0-a456-7b172fb81870 None admin] PATH_INFO: `/v3/users/819f4355d22549de99860edd3df36888` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:15.562559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e558c-52e6-43f0-a456-7b172fb81870 None admin] RBAC: Authorizing `identity:get_user(user_id=819f4355d22549de99860edd3df36888)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:15.562643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 27/52] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:15.564028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e558c-52e6-43f0-a456-7b172fb81870 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:15.566236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 26/53] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:15 2022] GET /identity/v3/users/819f4355d22549de99860edd3df36888 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:15.571533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4111940-9235-478b-a868-86ddc1b1582b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:15.586822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e558c-52e6-43f0-a456-7b172fb81870 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:15.675066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4111940-9235-478b-a868-86ddc1b1582b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:15.677412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:15.694325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e558c-52e6-43f0-a456-7b172fb81870 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:15.708935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d16dab-8642-4b79-a917-7be98fcc69dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:15.709742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d16dab-8642-4b79-a917-7be98fcc69dc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:15.710020 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d16dab-8642-4b79-a917-7be98fcc69dc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:15.710321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d16dab-8642-4b79-a917-7be98fcc69dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:15.711528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d16dab-8642-4b79-a917-7be98fcc69dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:15.713057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d16dab-8642-4b79-a917-7be98fcc69dc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:15.714694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 28/54] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:15.720446 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d16dab-8642-4b79-a917-7be98fcc69dc None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:15.727108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986c5ae-af6d-4fb6-be51-2bbcc28c0211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:15.727912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986c5ae-af6d-4fb6-be51-2bbcc28c0211 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:15.728228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986c5ae-af6d-4fb6-be51-2bbcc28c0211 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:15.728509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986c5ae-af6d-4fb6-be51-2bbcc28c0211 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:15.729682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986c5ae-af6d-4fb6-be51-2bbcc28c0211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:15.731137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986c5ae-af6d-4fb6-be51-2bbcc28c0211 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:15.732781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 27/55] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:15.738129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986c5ae-af6d-4fb6-be51-2bbcc28c0211 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:15.773362 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:15.821727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d16dab-8642-4b79-a917-7be98fcc69dc None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:15.837280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986c5ae-af6d-4fb6-be51-2bbcc28c0211 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:15.855362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d2a47-c124-45f1-b821-2a9a9f10f9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:15.856038 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d2a47-c124-45f1-b821-2a9a9f10f9c4 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:15.856352 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d2a47-c124-45f1-b821-2a9a9f10f9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:15.856632 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d2a47-c124-45f1-b821-2a9a9f10f9c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:15.857492 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d2a47-c124-45f1-b821-2a9a9f10f9c4 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.domain.id=default, user.id=819f4355d22549de99860edd3df36888) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:15.857815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d2a47-c124-45f1-b821-2a9a9f10f9c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:15.859282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d2a47-c124-45f1-b821-2a9a9f10f9c4 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:15.867278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 29/56] 10.222.0.20 () {56 vars in 1322 bytes} [Tue Jul 12 03:35:15 2022] GET /identity/v3/role_assignments?user.id=819f4355d22549de99860edd3df36888&scope.domain.id=default&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:15.869934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d7cac-db7a-49bd-8ffa-195c6b272020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:15.870628 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d7cac-db7a-49bd-8ffa-195c6b272020 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:15.870920 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d7cac-db7a-49bd-8ffa-195c6b272020 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:15.871203 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d7cac-db7a-49bd-8ffa-195c6b272020 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:15.871933 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d7cac-db7a-49bd-8ffa-195c6b272020 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:15.873482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d7cac-db7a-49bd-8ffa-195c6b272020 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:15.906353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 28/57] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Jul 12 03:35:15 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:35:17.957083 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186b07f-056e-4feb-8a84-ae7210058049 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:17.957637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186b07f-056e-4feb-8a84-ae7210058049 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:17.957637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186b07f-056e-4feb-8a84-ae7210058049 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:17.958985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 30/58] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:17.967458 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75933e2-dc16-4b86-84b6-09167a9e1757 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:17.967890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75933e2-dc16-4b86-84b6-09167a9e1757 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:17.968096 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75933e2-dc16-4b86-84b6-09167a9e1757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:17.993264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d75933e2-dc16-4b86-84b6-09167a9e1757 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:17.994515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d75933e2-dc16-4b86-84b6-09167a9e1757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:18.068910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 29/59] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:18.104690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186b07f-056e-4feb-8a84-ae7210058049 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:18.151710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:18.227291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186b07f-056e-4feb-8a84-ae7210058049 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:18.268039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b74344-ccfc-4788-ba86-19ce86b0fd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:18.268847 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b74344-ccfc-4788-ba86-19ce86b0fd16 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:18.269137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b74344-ccfc-4788-ba86-19ce86b0fd16 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:18.269421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b74344-ccfc-4788-ba86-19ce86b0fd16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:18.270772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b74344-ccfc-4788-ba86-19ce86b0fd16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:18.272270 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b74344-ccfc-4788-ba86-19ce86b0fd16 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:18.273881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 31/60] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:18.279369 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75933e2-dc16-4b86-84b6-09167a9e1757 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:18.299197 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:18.384179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75933e2-dc16-4b86-84b6-09167a9e1757 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:18.420250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bfd16-682a-44f8-8236-e61a3ca7b2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:18.420912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bfd16-682a-44f8-8236-e61a3ca7b2e3 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:18.421202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bfd16-682a-44f8-8236-e61a3ca7b2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:18.421476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bfd16-682a-44f8-8236-e61a3ca7b2e3 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:18.422183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bfd16-682a-44f8-8236-e61a3ca7b2e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:18.423716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bfd16-682a-44f8-8236-e61a3ca7b2e3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:18.450919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 30/61] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:35:18 2022] POST /identity/v3/users => generated 275 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:21.660311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a29ef1-4702-400a-9da5-fbe9a60d942e None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:21.660867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a29ef1-4702-400a-9da5-fbe9a60d942e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:21.660867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a29ef1-4702-400a-9da5-fbe9a60d942e None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:21.662508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 32/62] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:21.677816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfee656-d3e0-4453-a21f-2e22b1c84f5e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:21.678301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfee656-d3e0-4453-a21f-2e22b1c84f5e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:21.678596 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfee656-d3e0-4453-a21f-2e22b1c84f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:21.701016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-edfee656-d3e0-4453-a21f-2e22b1c84f5e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:21.702238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-edfee656-d3e0-4453-a21f-2e22b1c84f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:21.708838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b6836-d35c-4c6d-bf17-37f92994c417 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:21.709148 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b6836-d35c-4c6d-bf17-37f92994c417 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:21.709433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b6836-d35c-4c6d-bf17-37f92994c417 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:21.711049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 33/63] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:21.721057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3629a-1933-4579-9981-a3e127a6bc95 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:21.721353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3629a-1933-4579-9981-a3e127a6bc95 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:21.721639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3629a-1933-4579-9981-a3e127a6bc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:21.742974 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f9d3629a-1933-4579-9981-a3e127a6bc95 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:21.744225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f9d3629a-1933-4579-9981-a3e127a6bc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:21.806989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 31/64] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:21.808832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25a923-712d-4f7d-84c6-ff2a07972d9b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:21.809317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25a923-712d-4f7d-84c6-ff2a07972d9b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:21.809805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25a923-712d-4f7d-84c6-ff2a07972d9b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:21.811504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 32/65] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:21.817125 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3bac1-0194-4224-b95c-b92dc573ef9f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:21.817632 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3bac1-0194-4224-b95c-b92dc573ef9f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:21.818101 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3bac1-0194-4224-b95c-b92dc573ef9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:21.824439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:21.836612 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2ad3bac1-0194-4224-b95c-b92dc573ef9f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:21.838127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2ad3bac1-0194-4224-b95c-b92dc573ef9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:21.845706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 34/66] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:21.846655 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3629a-1933-4579-9981-a3e127a6bc95 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:21.879705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:21.923661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 33/67] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:21.932019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3bac1-0194-4224-b95c-b92dc573ef9f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:21.962712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3629a-1933-4579-9981-a3e127a6bc95 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:21.998070 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7f54e-bf7c-4849-97c1-116f18590592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:21.998907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7f54e-bf7c-4849-97c1-116f18590592 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:21.999176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7f54e-bf7c-4849-97c1-116f18590592 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:21.999458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7f54e-bf7c-4849-97c1-116f18590592 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.011223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c7f54e-bf7c-4849-97c1-116f18590592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-74c7f54e-bf7c-4849-97c1-116f18590592 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:22.020161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:22.021123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:22.021898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 35/68] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:35:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:22.023113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f18506-f94f-4028-97f7-1d504503ca34 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.023113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f18506-f94f-4028-97f7-1d504503ca34 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.023279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f18506-f94f-4028-97f7-1d504503ca34 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.024560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 36/69] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:22.025423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f18506-f94f-4028-97f7-1d504503ca34 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:22.047381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3bac1-0194-4224-b95c-b92dc573ef9f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:22.086064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493462f9-f2b9-42b9-8a58-ae9762bef3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:22.086981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493462f9-f2b9-42b9-8a58-ae9762bef3cc None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.087269 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493462f9-f2b9-42b9-8a58-ae9762bef3cc None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.087542 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493462f9-f2b9-42b9-8a58-ae9762bef3cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.089616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:22.099028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493462f9-f2b9-42b9-8a58-ae9762bef3cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-493462f9-f2b9-42b9-8a58-ae9762bef3cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:22.106803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:22.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:22.108496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 34/70] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:22.112069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41b22e-1e30-4a07-b4c9-3fc0b348d58a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.114572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41b22e-1e30-4a07-b4c9-3fc0b348d58a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.114864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41b22e-1e30-4a07-b4c9-3fc0b348d58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.133453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:22.134331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ed41b22e-1e30-4a07-b4c9-3fc0b348d58a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:22.135398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ed41b22e-1e30-4a07-b4c9-3fc0b348d58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:22.142333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f18506-f94f-4028-97f7-1d504503ca34 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:22.177579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caa668-f690-42ed-a1bc-26cf98fccdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:22.178328 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caa668-f690-42ed-a1bc-26cf98fccdb0 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.178616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caa668-f690-42ed-a1bc-26cf98fccdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.178891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caa668-f690-42ed-a1bc-26cf98fccdb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.189439 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94caa668-f690-42ed-a1bc-26cf98fccdb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-94caa668-f690-42ed-a1bc-26cf98fccdb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:22.197047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:22.197906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:22.198668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 37/71] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:22.203828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caa668-f690-42ed-a1bc-26cf98fccdb0 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:22.207809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 35/72] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:22.208747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41b22e-1e30-4a07-b4c9-3fc0b348d58a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:22.319929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41b22e-1e30-4a07-b4c9-3fc0b348d58a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:22.323200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caa668-f690-42ed-a1bc-26cf98fccdb0 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:22.347244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:22.357810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e8a7d-69f9-432e-9df3-b4a13c95b2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:22.358523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e8a7d-69f9-432e-9df3-b4a13c95b2b8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.358809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e8a7d-69f9-432e-9df3-b4a13c95b2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.359212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e8a7d-69f9-432e-9df3-b4a13c95b2b8 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.359971 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e8a7d-69f9-432e-9df3-b4a13c95b2b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:22.360347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e8a7d-69f9-432e-9df3-b4a13c95b2b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:22.360989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b9c45-cc93-4dad-a58f-180a529829ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:22.361666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b9c45-cc93-4dad-a58f-180a529829ee None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.361769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e8a7d-69f9-432e-9df3-b4a13c95b2b8 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:22.361955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b9c45-cc93-4dad-a58f-180a529829ee None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.362234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b9c45-cc93-4dad-a58f-180a529829ee None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.363159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b9c45-cc93-4dad-a58f-180a529829ee None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:22.363477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b9c45-cc93-4dad-a58f-180a529829ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:22.365251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b9c45-cc93-4dad-a58f-180a529829ee None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:22.375039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 38/73] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:22.375039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e8a7d-69f9-432e-9df3-b4a13c95b2b8 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:22.378491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 36/74] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:22.383087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b9c45-cc93-4dad-a58f-180a529829ee None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:22.394637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:22.483275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e8a7d-69f9-432e-9df3-b4a13c95b2b8 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:22.484968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b9c45-cc93-4dad-a58f-180a529829ee None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:22.515417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62767dac-9514-4ce0-8994-ee255b445d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:22.516118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62767dac-9514-4ce0-8994-ee255b445d6c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.516410 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62767dac-9514-4ce0-8994-ee255b445d6c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.516691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62767dac-9514-4ce0-8994-ee255b445d6c None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.517459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62767dac-9514-4ce0-8994-ee255b445d6c None admin] RBAC: Adding query filter params (name=member) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:22.517771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62767dac-9514-4ce0-8994-ee255b445d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:22.519193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62767dac-9514-4ce0-8994-ee255b445d6c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:22.520202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba89308-6ccf-4633-bf8a-e04387b9afed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:22.521049 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba89308-6ccf-4633-bf8a-e04387b9afed None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.521337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba89308-6ccf-4633-bf8a-e04387b9afed None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.521606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba89308-6ccf-4633-bf8a-e04387b9afed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.527809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 39/75] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:22.528569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62767dac-9514-4ce0-8994-ee255b445d6c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:22.536308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba89308-6ccf-4633-bf8a-e04387b9afed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-1ba89308-6ccf-4633-bf8a-e04387b9afed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:22.545836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:22.546702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:22.547447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 37/76] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:35:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:22.548117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba89308-6ccf-4633-bf8a-e04387b9afed None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:22.596906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:22.604594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:22.625975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62767dac-9514-4ce0-8994-ee255b445d6c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:22.651484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba89308-6ccf-4633-bf8a-e04387b9afed None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:22.659983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349e1dc-bd1f-4281-8873-938416f6bc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:22.660956 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349e1dc-bd1f-4281-8873-938416f6bc68 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.661980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349e1dc-bd1f-4281-8873-938416f6bc68 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.661980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349e1dc-bd1f-4281-8873-938416f6bc68 None admin] PATH_INFO: `/v3/users/819f4355d22549de99860edd3df36888` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.662703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349e1dc-bd1f-4281-8873-938416f6bc68 None admin] RBAC: Authorizing `identity:get_user(user_id=819f4355d22549de99860edd3df36888)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:22.664212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349e1dc-bd1f-4281-8873-938416f6bc68 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:22.665851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 40/77] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:22 2022] GET /identity/v3/users/819f4355d22549de99860edd3df36888 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:22.667324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f56b81-04c5-4814-a8d0-815f8f854da7 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.667613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f56b81-04c5-4814-a8d0-815f8f854da7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.667890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f56b81-04c5-4814-a8d0-815f8f854da7 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.672043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 41/78] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:22.672043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f56b81-04c5-4814-a8d0-815f8f854da7 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:22.689265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996393b-6952-4926-9d5c-827a79070f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:22.690204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996393b-6952-4926-9d5c-827a79070f00 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.690486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996393b-6952-4926-9d5c-827a79070f00 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.690763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2996393b-6952-4926-9d5c-827a79070f00 None admin] PATH_INFO: `/v3/users/ea069f8de53148a798edcfd4984feff3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.704388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2996393b-6952-4926-9d5c-827a79070f00 None admin] RBAC: Authorizing `identity:get_user(user_id=ea069f8de53148a798edcfd4984feff3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:22.705864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2996393b-6952-4926-9d5c-827a79070f00 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:22.707670 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 38/79] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:22 2022] GET /identity/v3/users/ea069f8de53148a798edcfd4984feff3 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:22.708288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996393b-6952-4926-9d5c-827a79070f00 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:22.771546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f56b81-04c5-4814-a8d0-815f8f854da7 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:22.804811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64705389-a8eb-4707-a6b3-4c4f7fd3a0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:22.805719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64705389-a8eb-4707-a6b3-4c4f7fd3a0d0 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.806006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64705389-a8eb-4707-a6b3-4c4f7fd3a0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.806288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64705389-a8eb-4707-a6b3-4c4f7fd3a0d0 None admin] PATH_INFO: `/v3/users/ea069f8de53148a798edcfd4984feff3` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.807500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64705389-a8eb-4707-a6b3-4c4f7fd3a0d0 None admin] RBAC: Authorizing `identity:get_user(user_id=ea069f8de53148a798edcfd4984feff3)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:22.807843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2996393b-6952-4926-9d5c-827a79070f00 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:22.809006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64705389-a8eb-4707-a6b3-4c4f7fd3a0d0 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:22.810235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:22.810896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 42/80] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:22 2022] GET /identity/v3/users/ea069f8de53148a798edcfd4984feff3 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:22.811661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64705389-a8eb-4707-a6b3-4c4f7fd3a0d0 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:22.827407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:22.846378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7791f-d841-412b-8565-5692dce09af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:22.847074 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7791f-d841-412b-8565-5692dce09af6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.847365 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7791f-d841-412b-8565-5692dce09af6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.847680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7791f-d841-412b-8565-5692dce09af6 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.848492 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7791f-d841-412b-8565-5692dce09af6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:22.848806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7791f-d841-412b-8565-5692dce09af6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:22.850221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7791f-d841-412b-8565-5692dce09af6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:22.858490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 39/81] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:35:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:22.859752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624adbf-5f74-4cc3-b75c-01287a8f4573 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.860080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624adbf-5f74-4cc3-b75c-01287a8f4573 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.860368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624adbf-5f74-4cc3-b75c-01287a8f4573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:22.877068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e624adbf-5f74-4cc3-b75c-01287a8f4573 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:22.878318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e624adbf-5f74-4cc3-b75c-01287a8f4573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:22.914939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64705389-a8eb-4707-a6b3-4c4f7fd3a0d0 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:22.945849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 40/82] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:22.946708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624adbf-5f74-4cc3-b75c-01287a8f4573 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:22.948479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bce01f-f867-4aba-a644-6b825dbbf564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:22.949150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bce01f-f867-4aba-a644-6b825dbbf564 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:22.949441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bce01f-f867-4aba-a644-6b825dbbf564 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:22.949728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bce01f-f867-4aba-a644-6b825dbbf564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:23.026182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bce01f-f867-4aba-a644-6b825dbbf564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:23.027604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bce01f-f867-4aba-a644-6b825dbbf564 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:23.039104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:23.043055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:23.046370 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624adbf-5f74-4cc3-b75c-01287a8f4573 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:23.083031 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257b8e4-8e0f-44f4-b110-ed93ef8178b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:23.083691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257b8e4-8e0f-44f4-b110-ed93ef8178b4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:23.083978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257b8e4-8e0f-44f4-b110-ed93ef8178b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:23.084294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257b8e4-8e0f-44f4-b110-ed93ef8178b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:23.126464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 43/83] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:23.127307 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bce01f-f867-4aba-a644-6b825dbbf564 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:23.154693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257b8e4-8e0f-44f4-b110-ed93ef8178b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:23.156262 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257b8e4-8e0f-44f4-b110-ed93ef8178b4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:23.226532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bce01f-f867-4aba-a644-6b825dbbf564 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:23.240181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:23.247704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:23.255303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 41/84] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:23.256129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257b8e4-8e0f-44f4-b110-ed93ef8178b4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:23.261574 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a9142-b185-4470-b29a-35c41977dab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:23.262233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a9142-b185-4470-b29a-35c41977dab5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:23.262528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a9142-b185-4470-b29a-35c41977dab5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:23.262813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a9142-b185-4470-b29a-35c41977dab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:23.333215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a9142-b185-4470-b29a-35c41977dab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:23.334719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a9142-b185-4470-b29a-35c41977dab5 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:23.354971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257b8e4-8e0f-44f4-b110-ed93ef8178b4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:23.389138 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ec996-f0fa-4688-986b-90d74e439320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:23.390079 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ec996-f0fa-4688-986b-90d74e439320 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:23.390378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ec996-f0fa-4688-986b-90d74e439320 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:23.390656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ec996-f0fa-4688-986b-90d74e439320 None admin] PATH_INFO: `/v3/users/ea069f8de53148a798edcfd4984feff3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:23.391877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ec996-f0fa-4688-986b-90d74e439320 None admin] RBAC: Authorizing `identity:get_user(user_id=ea069f8de53148a798edcfd4984feff3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:23.393423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ec996-f0fa-4688-986b-90d74e439320 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:23.395135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 42/85] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:23 2022] GET /identity/v3/users/ea069f8de53148a798edcfd4984feff3 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:23.395903 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ec996-f0fa-4688-986b-90d74e439320 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:23.433592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 44/86] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:23.434419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a9142-b185-4470-b29a-35c41977dab5 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:23.451937 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:23.460912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:23.498045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ec996-f0fa-4688-986b-90d74e439320 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:23.531164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189287d-5b6f-4437-8179-c15708f9b91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:23.531957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189287d-5b6f-4437-8179-c15708f9b91b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:23.532268 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189287d-5b6f-4437-8179-c15708f9b91b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:23.532544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189287d-5b6f-4437-8179-c15708f9b91b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:23.533752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189287d-5b6f-4437-8179-c15708f9b91b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:23.535234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189287d-5b6f-4437-8179-c15708f9b91b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:23.536735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a9142-b185-4470-b29a-35c41977dab5 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:23.537241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 43/87] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:23.537845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189287d-5b6f-4437-8179-c15708f9b91b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:23.578629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27008f3-c269-40ce-8f41-47e0419aff41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:23.579920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27008f3-c269-40ce-8f41-47e0419aff41 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:23.580239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27008f3-c269-40ce-8f41-47e0419aff41 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:23.580522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27008f3-c269-40ce-8f41-47e0419aff41 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:23.581828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27008f3-c269-40ce-8f41-47e0419aff41 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:23.583396 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27008f3-c269-40ce-8f41-47e0419aff41 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:23.585122 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 45/88] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:23 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:23.585889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27008f3-c269-40ce-8f41-47e0419aff41 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:23.639538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189287d-5b6f-4437-8179-c15708f9b91b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:23.664607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:23.670046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:23.675257 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfd413-eb92-4f80-bf5f-10e9c8ea2f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:23.676164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfd413-eb92-4f80-bf5f-10e9c8ea2f30 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:23.676449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfd413-eb92-4f80-bf5f-10e9c8ea2f30 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:23.676727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfd413-eb92-4f80-bf5f-10e9c8ea2f30 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:23.677915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfd413-eb92-4f80-bf5f-10e9c8ea2f30 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:23.679390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfd413-eb92-4f80-bf5f-10e9c8ea2f30 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:23.681160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 44/89] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:23 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:23.681991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfd413-eb92-4f80-bf5f-10e9c8ea2f30 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:23.683399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27008f3-c269-40ce-8f41-47e0419aff41 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:23.716865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342e1fc-0ca5-4f3c-834f-9ae6380c7662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:23.717595 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342e1fc-0ca5-4f3c-834f-9ae6380c7662 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:23.717893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342e1fc-0ca5-4f3c-834f-9ae6380c7662 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:23.718198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342e1fc-0ca5-4f3c-834f-9ae6380c7662 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:23.783420 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfd413-eb92-4f80-bf5f-10e9c8ea2f30 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:23.784418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342e1fc-0ca5-4f3c-834f-9ae6380c7662 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:23.785916 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342e1fc-0ca5-4f3c-834f-9ae6380c7662 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:23.817214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123d479-2da4-4029-995e-fc955a4920c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:23.818071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123d479-2da4-4029-995e-fc955a4920c3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:23.818353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123d479-2da4-4029-995e-fc955a4920c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:23.818629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123d479-2da4-4029-995e-fc955a4920c3 None admin] PATH_INFO: `/v3/projects/e442b172ca2f41339963bba006c17769` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:23.819815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123d479-2da4-4029-995e-fc955a4920c3 None admin] RBAC: Authorizing `identity:get_project(project_id=e442b172ca2f41339963bba006c17769)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:23.821305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123d479-2da4-4029-995e-fc955a4920c3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:23.823006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 45/90] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:23 2022] GET /identity/v3/projects/e442b172ca2f41339963bba006c17769 => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:23.823737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123d479-2da4-4029-995e-fc955a4920c3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:23.867841 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:23.887066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 46/91] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:23.888136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342e1fc-0ca5-4f3c-834f-9ae6380c7662 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:23.891065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:23.924998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123d479-2da4-4029-995e-fc955a4920c3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:23.965052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613055fb-5bf1-4bde-806f-6fd09265ca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:23.965743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613055fb-5bf1-4bde-806f-6fd09265ca33 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:23.966082 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613055fb-5bf1-4bde-806f-6fd09265ca33 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:23.966360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613055fb-5bf1-4bde-806f-6fd09265ca33 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:23.967129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613055fb-5bf1-4bde-806f-6fd09265ca33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:23.968740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613055fb-5bf1-4bde-806f-6fd09265ca33 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:23.985321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342e1fc-0ca5-4f3c-834f-9ae6380c7662 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:23.988310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 46/92] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:35:23 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:23.989093 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613055fb-5bf1-4bde-806f-6fd09265ca33 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:24.023380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9078f-4ac4-4955-8236-b9d068121db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:24.024234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9078f-4ac4-4955-8236-b9d068121db4 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:24.024573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9078f-4ac4-4955-8236-b9d068121db4 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:24.024956 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9078f-4ac4-4955-8236-b9d068121db4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:24.025859 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9078f-4ac4-4955-8236-b9d068121db4 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=b81760932eec4beda4343070aaf9b956, user.id=819f4355d22549de99860edd3df36888) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:24.026205 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9078f-4ac4-4955-8236-b9d068121db4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:24.027937 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9078f-4ac4-4955-8236-b9d068121db4 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:24.034738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 47/93] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:23 2022] GET /identity/v3/role_assignments?user.id=819f4355d22549de99860edd3df36888&scope.project.id=b81760932eec4beda4343070aaf9b956&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:24.041497 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9078f-4ac4-4955-8236-b9d068121db4 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:24.109969 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:24.136472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613055fb-5bf1-4bde-806f-6fd09265ca33 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:24.184594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9078f-4ac4-4955-8236-b9d068121db4 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:24.190626 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4735abb-4031-4c2b-a8a1-d51b820fb9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:24.191309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4735abb-4031-4c2b-a8a1-d51b820fb9c8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:24.191598 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4735abb-4031-4c2b-a8a1-d51b820fb9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:24.191880 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4735abb-4031-4c2b-a8a1-d51b820fb9c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:24.192774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4735abb-4031-4c2b-a8a1-d51b820fb9c8 None admin] RBAC: Adding query filter params (role.id=8f3ea95f038d4d8fbd1fd4897be36344, scope.project.id=b81760932eec4beda4343070aaf9b956, user.id=ea069f8de53148a798edcfd4984feff3) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:24.193089 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4735abb-4031-4c2b-a8a1-d51b820fb9c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:24.194704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4735abb-4031-4c2b-a8a1-d51b820fb9c8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:24.197563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:24.202985 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 47/94] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:23 2022] GET /identity/v3/role_assignments?user.id=ea069f8de53148a798edcfd4984feff3&scope.project.id=b81760932eec4beda4343070aaf9b956&role.id=8f3ea95f038d4d8fbd1fd4897be36344 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:24.209743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4735abb-4031-4c2b-a8a1-d51b820fb9c8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:24.235234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaf42d-9d4a-4528-8bcf-e588e609dc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:24.235929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaf42d-9d4a-4528-8bcf-e588e609dc89 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:24.236268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaf42d-9d4a-4528-8bcf-e588e609dc89 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:24.236554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaf42d-9d4a-4528-8bcf-e588e609dc89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:24.237419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaf42d-9d4a-4528-8bcf-e588e609dc89 None admin] RBAC: Adding query filter params (role.id=8f3ea95f038d4d8fbd1fd4897be36344, scope.project.id=e442b172ca2f41339963bba006c17769, user.id=ea069f8de53148a798edcfd4984feff3) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:24.237766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaf42d-9d4a-4528-8bcf-e588e609dc89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:24.239508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaf42d-9d4a-4528-8bcf-e588e609dc89 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:24.246371 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 48/95] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:24 2022] GET /identity/v3/role_assignments?user.id=ea069f8de53148a798edcfd4984feff3&scope.project.id=e442b172ca2f41339963bba006c17769&role.id=8f3ea95f038d4d8fbd1fd4897be36344 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:24.332907 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4735abb-4031-4c2b-a8a1-d51b820fb9c8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:24.374604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23490e61-30e2-426a-912d-1bb5e8176d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:24.375799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23490e61-30e2-426a-912d-1bb5e8176d0e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:24.376249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23490e61-30e2-426a-912d-1bb5e8176d0e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:24.376660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23490e61-30e2-426a-912d-1bb5e8176d0e None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:24.378223 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23490e61-30e2-426a-912d-1bb5e8176d0e None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:24.379900 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23490e61-30e2-426a-912d-1bb5e8176d0e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:24.382374 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 48/96] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:24 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:24.390354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaf42d-9d4a-4528-8bcf-e588e609dc89 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:24.515024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaf42d-9d4a-4528-8bcf-e588e609dc89 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:24.526151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:24.551060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e2177-217a-456e-a7f2-8d9da579c2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:24.551781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2177-217a-456e-a7f2-8d9da579c2cd None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:24.552105 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2177-217a-456e-a7f2-8d9da579c2cd None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:24.552389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2177-217a-456e-a7f2-8d9da579c2cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:24.553301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e2177-217a-456e-a7f2-8d9da579c2cd None admin] RBAC: Adding query filter params (role.id=ff6066b1fb8940cd9ec82c078fd8d2f6, scope.project.id=b81760932eec4beda4343070aaf9b956, user.id=ea069f8de53148a798edcfd4984feff3) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:24.553620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e2177-217a-456e-a7f2-8d9da579c2cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:24.555113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e2177-217a-456e-a7f2-8d9da579c2cd None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:24.561283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 49/97] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:24 2022] GET /identity/v3/role_assignments?user.id=ea069f8de53148a798edcfd4984feff3&scope.project.id=b81760932eec4beda4343070aaf9b956&role.id=ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:26.633775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4001ee-6eba-4ac9-8678-e77faadf772c None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:26.634851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4001ee-6eba-4ac9-8678-e77faadf772c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:26.635331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4001ee-6eba-4ac9-8678-e77faadf772c None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:26.636944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 49/98] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:26.642803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ed18f-8e74-4850-995a-557b96de8b32 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:26.643620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ed18f-8e74-4850-995a-557b96de8b32 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:26.644133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ed18f-8e74-4850-995a-557b96de8b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:26.664472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-3c5ed18f-8e74-4850-995a-557b96de8b32 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:26.665959 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3c5ed18f-8e74-4850-995a-557b96de8b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:26.696906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8cb64-461f-4234-b0ed-f367ec57461a None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:26.697513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8cb64-461f-4234-b0ed-f367ec57461a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:26.697978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8cb64-461f-4234-b0ed-f367ec57461a None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:26.699525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 50/99] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:26.718606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be95d59-5cb2-47be-8ed3-97b1baa9bf77 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:26.719224 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be95d59-5cb2-47be-8ed3-97b1baa9bf77 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:26.719692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be95d59-5cb2-47be-8ed3-97b1baa9bf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:26.748026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6be95d59-5cb2-47be-8ed3-97b1baa9bf77 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:26.748026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6be95d59-5cb2-47be-8ed3-97b1baa9bf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:26.802278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:26.803929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 50/100] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:26.822129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 51/101] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:26.854821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ed18f-8e74-4850-995a-557b96de8b32 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:26.885034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be95d59-5cb2-47be-8ed3-97b1baa9bf77 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:27.001196 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be95d59-5cb2-47be-8ed3-97b1baa9bf77 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:27.039478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dde78-8fce-4129-abeb-966f6bf44e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:27.040321 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dde78-8fce-4129-abeb-966f6bf44e70 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.040609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dde78-8fce-4129-abeb-966f6bf44e70 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.040890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dde78-8fce-4129-abeb-966f6bf44e70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.049102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ed18f-8e74-4850-995a-557b96de8b32 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:27.052300 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:27.065704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9dde78-8fce-4129-abeb-966f6bf44e70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-5e9dde78-8fce-4129-abeb-966f6bf44e70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:27.081750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:27.082649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:27.083466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 52/102] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:35:27.085347 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66634d64-d5b7-451c-a548-58c5939e3ab3 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.085843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66634d64-d5b7-451c-a548-58c5939e3ab3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.086342 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66634d64-d5b7-451c-a548-58c5939e3ab3 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.087923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 53/103] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:27.088859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66634d64-d5b7-451c-a548-58c5939e3ab3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:27.107659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147b2df-bfd2-456e-a32a-3b2723879bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:27.108826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147b2df-bfd2-456e-a32a-3b2723879bbe None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.109310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147b2df-bfd2-456e-a32a-3b2723879bbe None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.109772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147b2df-bfd2-456e-a32a-3b2723879bbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.125968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147b2df-bfd2-456e-a32a-3b2723879bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-f147b2df-bfd2-456e-a32a-3b2723879bbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:27.134721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:27.135618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:27.136443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:27.136443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:27.136443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:27.136443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:27.136443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:27.136443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:27.136443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:27.136443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:27.136443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:27.136443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:27.136443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:27.136443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:27.146464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 51/104] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:35:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:27.148787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614a054-bd6d-41e0-acdc-3761e389e14a None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.150068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614a054-bd6d-41e0-acdc-3761e389e14a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.152527 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614a054-bd6d-41e0-acdc-3761e389e14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.182523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-e614a054-bd6d-41e0-acdc-3761e389e14a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:27.184190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e614a054-bd6d-41e0-acdc-3761e389e14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:27.193436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:27.301742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66634d64-d5b7-451c-a548-58c5939e3ab3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:27.304613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 52/105] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:27.305483 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614a054-bd6d-41e0-acdc-3761e389e14a None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:27.367345 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e4790-9de7-4aaa-854a-b29c456b2477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:27.368029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e4790-9de7-4aaa-854a-b29c456b2477 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.368328 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e4790-9de7-4aaa-854a-b29c456b2477 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.368602 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e4790-9de7-4aaa-854a-b29c456b2477 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.369356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e4790-9de7-4aaa-854a-b29c456b2477 None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:27.369659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e4790-9de7-4aaa-854a-b29c456b2477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:27.371104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e4790-9de7-4aaa-854a-b29c456b2477 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:27.389704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 54/106] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:27.390473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e4790-9de7-4aaa-854a-b29c456b2477 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:27.470952 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:27.479939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614a054-bd6d-41e0-acdc-3761e389e14a None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:27.524581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf3da0-b7fd-44bb-97ad-c5ea531e5c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:27.525304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf3da0-b7fd-44bb-97ad-c5ea531e5c6e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.525594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf3da0-b7fd-44bb-97ad-c5ea531e5c6e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.525702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e4790-9de7-4aaa-854a-b29c456b2477 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:27.525883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf3da0-b7fd-44bb-97ad-c5ea531e5c6e None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.526661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf3da0-b7fd-44bb-97ad-c5ea531e5c6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:27.526976 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf3da0-b7fd-44bb-97ad-c5ea531e5c6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:27.529511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf3da0-b7fd-44bb-97ad-c5ea531e5c6e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:27.535910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:27.540442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 53/107] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:27.541399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf3da0-b7fd-44bb-97ad-c5ea531e5c6e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:27.566124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4b918-3687-417d-b22b-d173ba139cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:27.566941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4b918-3687-417d-b22b-d173ba139cd1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.567230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4b918-3687-417d-b22b-d173ba139cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.567610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4b918-3687-417d-b22b-d173ba139cd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.579172 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4b918-3687-417d-b22b-d173ba139cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-66f4b918-3687-417d-b22b-d173ba139cd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:27.587396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:27.588354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:27.589117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:27.589117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:27.589117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:27.589117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:27.589117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:27.589117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:27.589117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:27.589117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:27.589117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:27.589117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:27.589117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:27.589117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:27.593808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 55/108] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:27.595132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4067f-3188-4d08-a5a7-22685005648d None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.595425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4067f-3188-4d08-a5a7-22685005648d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.595704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4067f-3188-4d08-a5a7-22685005648d None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.597286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 56/109] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:27.601870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4067f-3188-4d08-a5a7-22685005648d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:27.650415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf3da0-b7fd-44bb-97ad-c5ea531e5c6e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:27.684507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb27ffb-7f47-4688-822a-945f2e64cd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:27.685384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb27ffb-7f47-4688-822a-945f2e64cd37 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.685668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb27ffb-7f47-4688-822a-945f2e64cd37 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.686332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb27ffb-7f47-4688-822a-945f2e64cd37 None admin] PATH_INFO: `/v3/users/ea069f8de53148a798edcfd4984feff3` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.687533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb27ffb-7f47-4688-822a-945f2e64cd37 None admin] RBAC: Authorizing `identity:get_user(user_id=ea069f8de53148a798edcfd4984feff3)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:27.689676 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb27ffb-7f47-4688-822a-945f2e64cd37 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:27.691475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 54/110] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:27 2022] GET /identity/v3/users/ea069f8de53148a798edcfd4984feff3 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:27.693293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb27ffb-7f47-4688-822a-945f2e64cd37 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:27.712673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4067f-3188-4d08-a5a7-22685005648d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:27.722967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:27.748136 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57a329-91b0-4e4f-82ac-85e763af9bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:27.749386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57a329-91b0-4e4f-82ac-85e763af9bd5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.749661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57a329-91b0-4e4f-82ac-85e763af9bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.749941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57a329-91b0-4e4f-82ac-85e763af9bd5 None admin] PATH_INFO: `/v3/users/819f4355d22549de99860edd3df36888` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.751144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57a329-91b0-4e4f-82ac-85e763af9bd5 None admin] RBAC: Authorizing `identity:get_user(user_id=819f4355d22549de99860edd3df36888)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:27.752655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57a329-91b0-4e4f-82ac-85e763af9bd5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:27.758971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 57/111] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:27 2022] GET /identity/v3/users/819f4355d22549de99860edd3df36888 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:27.758971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd854a37-8dc4-4252-8a76-6a21f22aa063 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.758971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd854a37-8dc4-4252-8a76-6a21f22aa063 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.758971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd854a37-8dc4-4252-8a76-6a21f22aa063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.772740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dd854a37-8dc4-4252-8a76-6a21f22aa063 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:27.773917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dd854a37-8dc4-4252-8a76-6a21f22aa063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:27.803246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:27.811111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb27ffb-7f47-4688-822a-945f2e64cd37 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:27.845348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 58/112] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:27.846553 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d97d2e-5089-42fe-af2a-4aca660e4c5c None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.846840 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d97d2e-5089-42fe-af2a-4aca660e4c5c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.847114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d97d2e-5089-42fe-af2a-4aca660e4c5c None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.850845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4ad92-2615-4091-8c97-f721d1756cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:27.850971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 59/113] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:27.851067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4ad92-2615-4091-8c97-f721d1756cd7 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.851151 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d97d2e-5089-42fe-af2a-4aca660e4c5c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:27.851225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4ad92-2615-4091-8c97-f721d1756cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.851225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4ad92-2615-4091-8c97-f721d1756cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.851225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4ad92-2615-4091-8c97-f721d1756cd7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:27.851225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4ad92-2615-4091-8c97-f721d1756cd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:27.852778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4ad92-2615-4091-8c97-f721d1756cd7 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:27.861765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 55/114] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:27.862870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4ad92-2615-4091-8c97-f721d1756cd7 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:27.947753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:27.956519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d97d2e-5089-42fe-af2a-4aca660e4c5c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:27.966339 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4ad92-2615-4091-8c97-f721d1756cd7 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:27.990809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7133eb-bb34-4ce6-bab7-e8cbfdd62eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:27.991674 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7133eb-bb34-4ce6-bab7-e8cbfdd62eb1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:27.991963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7133eb-bb34-4ce6-bab7-e8cbfdd62eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:27.992270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7133eb-bb34-4ce6-bab7-e8cbfdd62eb1 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:27.993445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7133eb-bb34-4ce6-bab7-e8cbfdd62eb1 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:27.994949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7133eb-bb34-4ce6-bab7-e8cbfdd62eb1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:27.999624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 60/115] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:27 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:27.999624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7133eb-bb34-4ce6-bab7-e8cbfdd62eb1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:28.002258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab0ad7-a06d-4983-969d-8b05a248053e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:28.003173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab0ad7-a06d-4983-969d-8b05a248053e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:28.003465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab0ad7-a06d-4983-969d-8b05a248053e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:28.003740 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab0ad7-a06d-4983-969d-8b05a248053e None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:28.004990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab0ad7-a06d-4983-969d-8b05a248053e None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:28.006436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab0ad7-a06d-4983-969d-8b05a248053e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:28.008174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 56/116] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:27 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:28.009354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac3f54-4e99-4cfb-aa35-61f69928cd04 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:28.009651 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac3f54-4e99-4cfb-aa35-61f69928cd04 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:28.009926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac3f54-4e99-4cfb-aa35-61f69928cd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:28.025871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-19ac3f54-4e99-4cfb-aa35-61f69928cd04 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:28.027158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-19ac3f54-4e99-4cfb-aa35-61f69928cd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:28.037878 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:28.095253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 57/117] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:28.097113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac3f54-4e99-4cfb-aa35-61f69928cd04 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:28.104642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7133eb-bb34-4ce6-bab7-e8cbfdd62eb1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:28.140273 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda3623-fd88-41ed-b9de-c52b5296e824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:28.141194 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda3623-fd88-41ed-b9de-c52b5296e824 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:28.141480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda3623-fd88-41ed-b9de-c52b5296e824 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:28.141759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda3623-fd88-41ed-b9de-c52b5296e824 None admin] PATH_INFO: `/v3/users/ea069f8de53148a798edcfd4984feff3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:28.143007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda3623-fd88-41ed-b9de-c52b5296e824 None admin] RBAC: Authorizing `identity:get_user(user_id=ea069f8de53148a798edcfd4984feff3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:28.144542 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda3623-fd88-41ed-b9de-c52b5296e824 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:28.146210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 61/118] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:27 2022] GET /identity/v3/users/ea069f8de53148a798edcfd4984feff3 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:28.147412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af7b6d-0698-4133-b031-53c6639d0543 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:28.147650 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af7b6d-0698-4133-b031-53c6639d0543 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:28.147962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af7b6d-0698-4133-b031-53c6639d0543 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:28.149187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 62/119] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:28.149859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af7b6d-0698-4133-b031-53c6639d0543 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:28.186296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:28.202059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac3f54-4e99-4cfb-aa35-61f69928cd04 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:28.234485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ff73c-50ce-4036-8265-dbcce187135d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:28.235286 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ff73c-50ce-4036-8265-dbcce187135d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:28.235570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ff73c-50ce-4036-8265-dbcce187135d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:28.235847 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ff73c-50ce-4036-8265-dbcce187135d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:28.237063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ff73c-50ce-4036-8265-dbcce187135d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:28.238545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ff73c-50ce-4036-8265-dbcce187135d None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:28.240163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 58/120] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:28.240888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ff73c-50ce-4036-8265-dbcce187135d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:28.253453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af7b6d-0698-4133-b031-53c6639d0543 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:28.255057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:28.287767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a36dd-e3bf-40ae-a466-489d53270425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:28.288711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a36dd-e3bf-40ae-a466-489d53270425 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:28.288999 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a36dd-e3bf-40ae-a466-489d53270425 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:28.289280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a36dd-e3bf-40ae-a466-489d53270425 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956/users/ea069f8de53148a798edcfd4984feff3/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:28.291256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a36dd-e3bf-40ae-a466-489d53270425 None admin] RBAC: Authorizing `identity:create_grant(project_id=b81760932eec4beda4343070aaf9b956, user_id=ea069f8de53148a798edcfd4984feff3, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:28.292814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a36dd-e3bf-40ae-a466-489d53270425 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:28.306135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 63/121] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:35:28 2022] PUT /identity/v3/projects/b81760932eec4beda4343070aaf9b956/users/ea069f8de53148a798edcfd4984feff3/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:35:28.306917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a36dd-e3bf-40ae-a466-489d53270425 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:28.345280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ff73c-50ce-4036-8265-dbcce187135d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:28.384284 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc402f2-a12e-4c69-83ee-7e5892074015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:28.385242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc402f2-a12e-4c69-83ee-7e5892074015 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:28.385526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc402f2-a12e-4c69-83ee-7e5892074015 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:28.385800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc402f2-a12e-4c69-83ee-7e5892074015 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956/users/819f4355d22549de99860edd3df36888/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:28.388050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc402f2-a12e-4c69-83ee-7e5892074015 None admin] RBAC: Authorizing `identity:create_grant(project_id=b81760932eec4beda4343070aaf9b956, user_id=819f4355d22549de99860edd3df36888, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:28.389599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc402f2-a12e-4c69-83ee-7e5892074015 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:28.416036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 59/122] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:35:28 2022] PUT /identity/v3/projects/b81760932eec4beda4343070aaf9b956/users/819f4355d22549de99860edd3df36888/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:35:28.416036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc402f2-a12e-4c69-83ee-7e5892074015 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:28.423807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:28.455808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a36dd-e3bf-40ae-a466-489d53270425 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:28.504051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8cd43-1809-49f8-8b21-99dc016b652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:28.504875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8cd43-1809-49f8-8b21-99dc016b652b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:28.505166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8cd43-1809-49f8-8b21-99dc016b652b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:28.505441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8cd43-1809-49f8-8b21-99dc016b652b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:28.519400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8cd43-1809-49f8-8b21-99dc016b652b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-64c8cd43-1809-49f8-8b21-99dc016b652b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:28.527662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:28.528622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:28.529355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 64/123] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:28.535003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cb791-06e5-46df-adba-4b1076036707 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:28.535294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cb791-06e5-46df-adba-4b1076036707 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:28.535584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cb791-06e5-46df-adba-4b1076036707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:28.547630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc402f2-a12e-4c69-83ee-7e5892074015 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:28.556272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4d9cb791-06e5-46df-adba-4b1076036707 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:28.558346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4d9cb791-06e5-46df-adba-4b1076036707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:28.586644 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:28.590721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfffc95-05eb-4988-a4d6-01b02e05a152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:28.591627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfffc95-05eb-4988-a4d6-01b02e05a152 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:28.591917 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfffc95-05eb-4988-a4d6-01b02e05a152 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:28.592435 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfffc95-05eb-4988-a4d6-01b02e05a152 None admin] PATH_INFO: `/v3/projects/e442b172ca2f41339963bba006c17769` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:28.593892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfffc95-05eb-4988-a4d6-01b02e05a152 None admin] RBAC: Authorizing `identity:get_project(project_id=e442b172ca2f41339963bba006c17769)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:28.595198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfffc95-05eb-4988-a4d6-01b02e05a152 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:28.597394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 60/124] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:28 2022] GET /identity/v3/projects/e442b172ca2f41339963bba006c17769 => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:28.598177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfffc95-05eb-4988-a4d6-01b02e05a152 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:28.647438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 65/125] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:28.648465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cb791-06e5-46df-adba-4b1076036707 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:28.709484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:28.731092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfffc95-05eb-4988-a4d6-01b02e05a152 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:28.766863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074904f8-b3ab-4bb1-9d50-59a2796cd7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:28.767669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074904f8-b3ab-4bb1-9d50-59a2796cd7fa None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:28.768129 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074904f8-b3ab-4bb1-9d50-59a2796cd7fa None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:28.768509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074904f8-b3ab-4bb1-9d50-59a2796cd7fa None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:28.769318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074904f8-b3ab-4bb1-9d50-59a2796cd7fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:28.769847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cb791-06e5-46df-adba-4b1076036707 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:28.770995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074904f8-b3ab-4bb1-9d50-59a2796cd7fa None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:28.800621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 61/126] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:35:28 2022] POST /identity/v3/projects => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:28.809231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074904f8-b3ab-4bb1-9d50-59a2796cd7fa None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:28.811088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc17d52-cad9-4f7b-92db-d5e1e91db731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:28.811943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc17d52-cad9-4f7b-92db-d5e1e91db731 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:28.812549 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc17d52-cad9-4f7b-92db-d5e1e91db731 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:28.812963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc17d52-cad9-4f7b-92db-d5e1e91db731 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:28.814021 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc17d52-cad9-4f7b-92db-d5e1e91db731 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:28.814544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc17d52-cad9-4f7b-92db-d5e1e91db731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:28.816205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc17d52-cad9-4f7b-92db-d5e1e91db731 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:28.827483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 66/127] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:28.828901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc17d52-cad9-4f7b-92db-d5e1e91db731 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:28.852398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:28.968398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074904f8-b3ab-4bb1-9d50-59a2796cd7fa None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:28.982844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc17d52-cad9-4f7b-92db-d5e1e91db731 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:29.000434 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:29.007319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f56a84-9e47-4bdb-bd1c-8097ad14ca25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:29.014776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f56a84-9e47-4bdb-bd1c-8097ad14ca25 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:29.014776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f56a84-9e47-4bdb-bd1c-8097ad14ca25 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:29.014776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f56a84-9e47-4bdb-bd1c-8097ad14ca25 None admin] PATH_INFO: `/v3/projects/e442b172ca2f41339963bba006c17769/users/ea069f8de53148a798edcfd4984feff3/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:29.025780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f56a84-9e47-4bdb-bd1c-8097ad14ca25 None admin] RBAC: Authorizing `identity:create_grant(project_id=e442b172ca2f41339963bba006c17769, user_id=ea069f8de53148a798edcfd4984feff3, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:29.027367 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f56a84-9e47-4bdb-bd1c-8097ad14ca25 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:29.044503 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cf285-065d-40d3-bff5-eb9ec454604c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:29.049074 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cf285-065d-40d3-bff5-eb9ec454604c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:29.053858 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 62/128] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:35:28 2022] PUT /identity/v3/projects/e442b172ca2f41339963bba006c17769/users/ea069f8de53148a798edcfd4984feff3/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:35:29.055041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f56a84-9e47-4bdb-bd1c-8097ad14ca25 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:29.057463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cf285-065d-40d3-bff5-eb9ec454604c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:29.057994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cf285-065d-40d3-bff5-eb9ec454604c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:29.082786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104cf285-065d-40d3-bff5-eb9ec454604c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-104cf285-065d-40d3-bff5-eb9ec454604c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:29.099505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:29.100531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:29.101280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:29.101280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:29.101280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:29.101280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:29.101280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:29.101280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:29.101280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:29.101280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:29.101280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:29.101280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:29.101280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:29.101280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:29.107045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 67/129] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:35:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:29.113046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cf285-065d-40d3-bff5-eb9ec454604c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:29.287646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:29.292052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f56a84-9e47-4bdb-bd1c-8097ad14ca25 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:29.311906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cf285-065d-40d3-bff5-eb9ec454604c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:29.337571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcc86a-381b-4911-974e-6f23ce09a5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:29.338490 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcc86a-381b-4911-974e-6f23ce09a5f5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:29.338815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcc86a-381b-4911-974e-6f23ce09a5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:29.339100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcc86a-381b-4911-974e-6f23ce09a5f5 None admin] PATH_INFO: `/v3/users/4aa94cd636fc48ed938817335b7870ca` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:29.354401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890f8a2-6a40-4594-b7d9-2fb5df7df1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:29.355422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890f8a2-6a40-4594-b7d9-2fb5df7df1cd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:29.355917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890f8a2-6a40-4594-b7d9-2fb5df7df1cd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:29.356416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890f8a2-6a40-4594-b7d9-2fb5df7df1cd None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:29.357372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890f8a2-6a40-4594-b7d9-2fb5df7df1cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:29.357969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890f8a2-6a40-4594-b7d9-2fb5df7df1cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:29.359582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890f8a2-6a40-4594-b7d9-2fb5df7df1cd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:29.365071 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dcc86a-381b-4911-974e-6f23ce09a5f5 None admin] RBAC: Authorizing `identity:get_user(user_id=4aa94cd636fc48ed938817335b7870ca)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:29.366547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dcc86a-381b-4911-974e-6f23ce09a5f5 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:29.368529 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 63/130] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:29 2022] GET /identity/v3/users/4aa94cd636fc48ed938817335b7870ca => generated 289 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:29.369767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 68/131] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:35:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:29.375338 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcc86a-381b-4911-974e-6f23ce09a5f5 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:29.376029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890f8a2-6a40-4594-b7d9-2fb5df7df1cd None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:29.444714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:29.534420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcc86a-381b-4911-974e-6f23ce09a5f5 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:29.558107 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890f8a2-6a40-4594-b7d9-2fb5df7df1cd None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:29.599018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3673bd-f332-4afe-b52f-0eea76fce803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:29.600796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3673bd-f332-4afe-b52f-0eea76fce803 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:29.601459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3673bd-f332-4afe-b52f-0eea76fce803 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:29.603104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3673bd-f332-4afe-b52f-0eea76fce803 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:29.623900 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dd22e-4024-406b-80ed-c75c56a62126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:29.626420 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dd22e-4024-406b-80ed-c75c56a62126 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:29.627158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dd22e-4024-406b-80ed-c75c56a62126 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:29.628331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dd22e-4024-406b-80ed-c75c56a62126 None admin] PATH_INFO: `/v3/users/ea069f8de53148a798edcfd4984feff3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:29.631260 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dd22e-4024-406b-80ed-c75c56a62126 None admin] RBAC: Authorizing `identity:get_user(user_id=ea069f8de53148a798edcfd4984feff3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:29.633208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dd22e-4024-406b-80ed-c75c56a62126 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:29.636811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 69/132] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:29 2022] GET /identity/v3/users/ea069f8de53148a798edcfd4984feff3 => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:29.643060 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dd22e-4024-406b-80ed-c75c56a62126 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:29.676377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:29.728059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3673bd-f332-4afe-b52f-0eea76fce803 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:29.729629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3673bd-f332-4afe-b52f-0eea76fce803 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:29.794549 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dd22e-4024-406b-80ed-c75c56a62126 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:29.814866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:29.831748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566820c5-ba21-4021-8702-9f193085087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:29.832742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566820c5-ba21-4021-8702-9f193085087e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:29.833125 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566820c5-ba21-4021-8702-9f193085087e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:29.833407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566820c5-ba21-4021-8702-9f193085087e None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:29.834610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566820c5-ba21-4021-8702-9f193085087e None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:29.836101 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566820c5-ba21-4021-8702-9f193085087e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:29.837999 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 70/133] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:29 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:29.843843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566820c5-ba21-4021-8702-9f193085087e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:29.879306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 64/134] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:29.885292 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3673bd-f332-4afe-b52f-0eea76fce803 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:29.975911 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:30.032549 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566820c5-ba21-4021-8702-9f193085087e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:30.054448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3673bd-f332-4afe-b52f-0eea76fce803 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:30.108849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969bb5c3-fc0e-4130-ace0-ed4a95191cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:30.109691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bb5c3-fc0e-4130-ace0-ed4a95191cb3 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:30.109980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bb5c3-fc0e-4130-ace0-ed4a95191cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:30.110260 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969bb5c3-fc0e-4130-ace0-ed4a95191cb3 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956/users/ea069f8de53148a798edcfd4984feff3/roles/ff6066b1fb8940cd9ec82c078fd8d2f6` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:30.114007 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c1380-6f64-41d5-b394-517a7b996386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:30.114883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c1380-6f64-41d5-b394-517a7b996386 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:30.115173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c1380-6f64-41d5-b394-517a7b996386 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:30.115453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c1380-6f64-41d5-b394-517a7b996386 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:30.116796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bb5c3-fc0e-4130-ace0-ed4a95191cb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b81760932eec4beda4343070aaf9b956, user_id=ea069f8de53148a798edcfd4984feff3, role_id=ff6066b1fb8940cd9ec82c078fd8d2f6)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:30.118301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969bb5c3-fc0e-4130-ace0-ed4a95191cb3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:30.120754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c1380-6f64-41d5-b394-517a7b996386 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:30.122290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c1380-6f64-41d5-b394-517a7b996386 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:30.124691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 65/135] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:29 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:30.130486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c1380-6f64-41d5-b394-517a7b996386 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:30.137875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 71/136] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:35:29 2022] PUT /identity/v3/projects/b81760932eec4beda4343070aaf9b956/users/ea069f8de53148a798edcfd4984feff3/roles/ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:35:30.176408 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:30.293245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c1380-6f64-41d5-b394-517a7b996386 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:30.354160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70448e2-a74e-433c-ae98-f7be233c800b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:30.355188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70448e2-a74e-433c-ae98-f7be233c800b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:30.355663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70448e2-a74e-433c-ae98-f7be233c800b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:30.356177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70448e2-a74e-433c-ae98-f7be233c800b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:30.357210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70448e2-a74e-433c-ae98-f7be233c800b None admin] RBAC: Adding query filter params (role.id=f740696dd1a847929b6236dcca34cc2f, scope.project.id=b81760932eec4beda4343070aaf9b956, user.id=4aa94cd636fc48ed938817335b7870ca) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:30.357716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70448e2-a74e-433c-ae98-f7be233c800b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:30.359468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70448e2-a74e-433c-ae98-f7be233c800b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:30.368754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 66/137] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:30 2022] GET /identity/v3/role_assignments?user.id=4aa94cd636fc48ed938817335b7870ca&scope.project.id=b81760932eec4beda4343070aaf9b956&role.id=f740696dd1a847929b6236dcca34cc2f => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:30.924372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef973293-f125-4847-8157-fa686b60ae89 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:30.924899 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef973293-f125-4847-8157-fa686b60ae89 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:30.924969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef973293-f125-4847-8157-fa686b60ae89 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:30.926642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 72/138] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:30.931873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59551e34-3362-41c1-8f48-b07b87de28dc None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:30.932333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59551e34-3362-41c1-8f48-b07b87de28dc None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:30.932520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59551e34-3362-41c1-8f48-b07b87de28dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:30.959159 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-59551e34-3362-41c1-8f48-b07b87de28dc None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:30.959337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-59551e34-3362-41c1-8f48-b07b87de28dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:31.055925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 67/139] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:31.092410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef973293-f125-4847-8157-fa686b60ae89 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:31.142127 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12546124-0ed8-42b5-bf5b-19ba76a4b531 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:31.142441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12546124-0ed8-42b5-bf5b-19ba76a4b531 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:31.142696 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12546124-0ed8-42b5-bf5b-19ba76a4b531 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:31.144098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 68/140] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:31.149315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3f8ce-36a1-4627-ae11-e965d937eb1f None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:31.149627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3f8ce-36a1-4627-ae11-e965d937eb1f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:31.149926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3f8ce-36a1-4627-ae11-e965d937eb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:31.177853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-ebd3f8ce-36a1-4627-ae11-e965d937eb1f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:31.178996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ebd3f8ce-36a1-4627-ae11-e965d937eb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:31.194598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:31.282147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef973293-f125-4847-8157-fa686b60ae89 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:31.321911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 69/141] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:31.334910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:31.344655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a819e5-e8c7-4661-9246-c46b69f8f357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:31.345484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a819e5-e8c7-4661-9246-c46b69f8f357 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:31.345823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a819e5-e8c7-4661-9246-c46b69f8f357 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:31.346335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a819e5-e8c7-4661-9246-c46b69f8f357 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:31.361296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3f8ce-36a1-4627-ae11-e965d937eb1f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:31.375800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a819e5-e8c7-4661-9246-c46b69f8f357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e9a819e5-e8c7-4661-9246-c46b69f8f357 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:31.383660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:31.384787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:31.385579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 73/142] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:31.401276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a819e5-e8c7-4661-9246-c46b69f8f357 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:31.554019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3f8ce-36a1-4627-ae11-e965d937eb1f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:31.616426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5996d-b2de-421c-88bc-011e6d299d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:31.617294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5996d-b2de-421c-88bc-011e6d299d10 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:31.617634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5996d-b2de-421c-88bc-011e6d299d10 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:31.617914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5996d-b2de-421c-88bc-011e6d299d10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:31.637224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5996d-b2de-421c-88bc-011e6d299d10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:31.646091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a819e5-e8c7-4661-9246-c46b69f8f357 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-bda5996d-b2de-421c-88bc-011e6d299d10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:31.647784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:31.648661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:31.649561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 70/143] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:35:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:35:31.656986 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5996d-b2de-421c-88bc-011e6d299d10 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:31.660660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:31.760739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9bbc41-7539-4ddd-b34f-4d35f812081e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:31.761489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bbc41-7539-4ddd-b34f-4d35f812081e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:31.761818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bbc41-7539-4ddd-b34f-4d35f812081e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:31.762076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9bbc41-7539-4ddd-b34f-4d35f812081e None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:31.763142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9bbc41-7539-4ddd-b34f-4d35f812081e None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:31.763671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9bbc41-7539-4ddd-b34f-4d35f812081e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:31.765815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9bbc41-7539-4ddd-b34f-4d35f812081e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:31.786924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 74/144] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:31.792507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9bbc41-7539-4ddd-b34f-4d35f812081e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:31.848058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5996d-b2de-421c-88bc-011e6d299d10 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:31.928360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:31.930158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036dbdc-57d6-4c8a-9758-d0b467fea9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:31.931131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036dbdc-57d6-4c8a-9758-d0b467fea9da None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:31.931529 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036dbdc-57d6-4c8a-9758-d0b467fea9da None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:31.932081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036dbdc-57d6-4c8a-9758-d0b467fea9da None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:31.933015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036dbdc-57d6-4c8a-9758-d0b467fea9da None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:31.933495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036dbdc-57d6-4c8a-9758-d0b467fea9da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:31.935054 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036dbdc-57d6-4c8a-9758-d0b467fea9da None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:31.952126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 71/145] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:31.963166 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036dbdc-57d6-4c8a-9758-d0b467fea9da None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:32.003596 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9bbc41-7539-4ddd-b34f-4d35f812081e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:32.065059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c6389-dfb2-47c5-be48-ede308133bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:32.066407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c6389-dfb2-47c5-be48-ede308133bd5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:32.067461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c6389-dfb2-47c5-be48-ede308133bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:32.068069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c6389-dfb2-47c5-be48-ede308133bd5 None admin] PATH_INFO: `/v3/users/ea069f8de53148a798edcfd4984feff3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:32.070786 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:32.072195 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c6389-dfb2-47c5-be48-ede308133bd5 None admin] RBAC: Authorizing `identity:get_user(user_id=ea069f8de53148a798edcfd4984feff3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:32.074619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c6389-dfb2-47c5-be48-ede308133bd5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:32.079074 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 75/146] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:31 2022] GET /identity/v3/users/ea069f8de53148a798edcfd4984feff3 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:32.080763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ece8ad-4885-4a6e-aaab-c3416c2b8086 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:32.082363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ece8ad-4885-4a6e-aaab-c3416c2b8086 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:32.082839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ece8ad-4885-4a6e-aaab-c3416c2b8086 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:32.085595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 76/147] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:32.086915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ece8ad-4885-4a6e-aaab-c3416c2b8086 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:32.132228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036dbdc-57d6-4c8a-9758-d0b467fea9da None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:32.170659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48e1b5-45ad-4953-8052-05826bd5ed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:32.171556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48e1b5-45ad-4953-8052-05826bd5ed31 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:32.171847 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48e1b5-45ad-4953-8052-05826bd5ed31 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:32.172174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48e1b5-45ad-4953-8052-05826bd5ed31 None admin] PATH_INFO: `/v3/users/819f4355d22549de99860edd3df36888` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:32.173393 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48e1b5-45ad-4953-8052-05826bd5ed31 None admin] RBAC: Authorizing `identity:get_user(user_id=819f4355d22549de99860edd3df36888)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:32.174863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48e1b5-45ad-4953-8052-05826bd5ed31 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:32.176690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 72/148] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:31 2022] GET /identity/v3/users/819f4355d22549de99860edd3df36888 => generated 247 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:32.177910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a92af-9cf4-445f-baad-048acf2d75e1 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:32.178198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a92af-9cf4-445f-baad-048acf2d75e1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:32.178475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a92af-9cf4-445f-baad-048acf2d75e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:32.199211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d58a92af-9cf4-445f-baad-048acf2d75e1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:32.200448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d58a92af-9cf4-445f-baad-048acf2d75e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:32.271392 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 73/149] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:32.272678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a92af-9cf4-445f-baad-048acf2d75e1 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:32.296998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ece8ad-4885-4a6e-aaab-c3416c2b8086 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:32.320857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:32.333784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459b0f8-0762-42df-8bed-6f379f3bfbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:32.334450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459b0f8-0762-42df-8bed-6f379f3bfbe5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:32.334734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459b0f8-0762-42df-8bed-6f379f3bfbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:32.335009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459b0f8-0762-42df-8bed-6f379f3bfbe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:32.345899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:32.393672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a92af-9cf4-445f-baad-048acf2d75e1 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:32.407617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459b0f8-0762-42df-8bed-6f379f3bfbe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:32.409233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7459b0f8-0762-42df-8bed-6f379f3bfbe5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:32.435127 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183cb8a-87e5-4f8e-b59f-a508a26392a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:32.436118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183cb8a-87e5-4f8e-b59f-a508a26392a0 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:32.436666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183cb8a-87e5-4f8e-b59f-a508a26392a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:32.436952 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183cb8a-87e5-4f8e-b59f-a508a26392a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:32.513575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183cb8a-87e5-4f8e-b59f-a508a26392a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:32.514751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 77/150] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:32.515409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183cb8a-87e5-4f8e-b59f-a508a26392a0 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:32.520780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ae006-dacf-4cd7-a972-3f47cb60fb5f None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:32.521069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ae006-dacf-4cd7-a972-3f47cb60fb5f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:32.521356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ae006-dacf-4cd7-a972-3f47cb60fb5f None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:32.522727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 78/151] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:32.523437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ae006-dacf-4cd7-a972-3f47cb60fb5f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:32.554698 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:32.578814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:32.619589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 74/152] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:32.620640 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183cb8a-87e5-4f8e-b59f-a508a26392a0 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:32.644372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ae006-dacf-4cd7-a972-3f47cb60fb5f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:32.685573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82321edf-744b-4fe3-8f28-866b6184859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:32.686422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82321edf-744b-4fe3-8f28-866b6184859a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:32.686716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82321edf-744b-4fe3-8f28-866b6184859a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:32.686998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82321edf-744b-4fe3-8f28-866b6184859a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:32.688382 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82321edf-744b-4fe3-8f28-866b6184859a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:32.690022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82321edf-744b-4fe3-8f28-866b6184859a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:32.692134 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 79/153] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:32.693404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e4175-b952-49c2-a8ca-117f9465889e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:32.693692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e4175-b952-49c2-a8ca-117f9465889e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:32.693967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e4175-b952-49c2-a8ca-117f9465889e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:32.712428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-313e4175-b952-49c2-a8ca-117f9465889e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:32.713679 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-313e4175-b952-49c2-a8ca-117f9465889e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:32.724459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183cb8a-87e5-4f8e-b59f-a508a26392a0 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:32.759569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c56bd-39b0-4068-bcb4-3588e970bb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:32.760541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c56bd-39b0-4068-bcb4-3588e970bb93 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:32.760834 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c56bd-39b0-4068-bcb4-3588e970bb93 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:32.761107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c56bd-39b0-4068-bcb4-3588e970bb93 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:32.762453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c56bd-39b0-4068-bcb4-3588e970bb93 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:32.764032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c56bd-39b0-4068-bcb4-3588e970bb93 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:32.766063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 75/154] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:32 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:32.766621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c56bd-39b0-4068-bcb4-3588e970bb93 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:32.804099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 80/155] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:32.804988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e4175-b952-49c2-a8ca-117f9465889e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:32.822409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:32.842834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:32.897864 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c56bd-39b0-4068-bcb4-3588e970bb93 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:32.932288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed6d30-626b-4deb-8b6a-9b76718f5d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:32.933065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e4175-b952-49c2-a8ca-117f9465889e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:32.933498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed6d30-626b-4deb-8b6a-9b76718f5d13 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:32.933498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed6d30-626b-4deb-8b6a-9b76718f5d13 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:32.933795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed6d30-626b-4deb-8b6a-9b76718f5d13 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:32.934935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed6d30-626b-4deb-8b6a-9b76718f5d13 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:32.936650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed6d30-626b-4deb-8b6a-9b76718f5d13 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:32.938855 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 76/156] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:32 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:32.939805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed6d30-626b-4deb-8b6a-9b76718f5d13 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:32.970781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387b925-e1a8-40c7-ad01-2712ef7fb56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:32.971571 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387b925-e1a8-40c7-ad01-2712ef7fb56b None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:32.971919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387b925-e1a8-40c7-ad01-2712ef7fb56b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:32.972405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387b925-e1a8-40c7-ad01-2712ef7fb56b None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:32.973476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387b925-e1a8-40c7-ad01-2712ef7fb56b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:32.975158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387b925-e1a8-40c7-ad01-2712ef7fb56b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:32.998581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 81/157] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:35:32 2022] POST /identity/v3/users => generated 283 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:32.999993 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387b925-e1a8-40c7-ad01-2712ef7fb56b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:33.047120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:33.051524 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed6d30-626b-4deb-8b6a-9b76718f5d13 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:33.090259 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7fc49-b562-43ac-9121-67003c92cd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:33.091491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7fc49-b562-43ac-9121-67003c92cd86 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:33.091978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7fc49-b562-43ac-9121-67003c92cd86 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:33.092485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7fc49-b562-43ac-9121-67003c92cd86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:33.093582 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7fc49-b562-43ac-9121-67003c92cd86 None admin] RBAC: Adding query filter params (role.id=8f3ea95f038d4d8fbd1fd4897be36344, scope.project.id=b81760932eec4beda4343070aaf9b956, user.id=ea069f8de53148a798edcfd4984feff3) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:33.094130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7fc49-b562-43ac-9121-67003c92cd86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:33.095931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:33.096227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7fc49-b562-43ac-9121-67003c92cd86 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:33.103765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 77/158] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:32 2022] GET /identity/v3/role_assignments?user.id=ea069f8de53148a798edcfd4984feff3&scope.project.id=b81760932eec4beda4343070aaf9b956&role.id=8f3ea95f038d4d8fbd1fd4897be36344 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:33.105029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7fc49-b562-43ac-9121-67003c92cd86 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:33.137464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387b925-e1a8-40c7-ad01-2712ef7fb56b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:33.202319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a350e-b110-47be-9ed9-463a21ecdae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:33.203565 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a350e-b110-47be-9ed9-463a21ecdae5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:33.203846 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a350e-b110-47be-9ed9-463a21ecdae5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:33.204185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a350e-b110-47be-9ed9-463a21ecdae5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:33.221373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a350e-b110-47be-9ed9-463a21ecdae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-967a350e-b110-47be-9ed9-463a21ecdae5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:33.230288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:33.231256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:33.232474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 82/159] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:33.233132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5df57f-9d4f-4bfb-bdb9-ec4e4bf30258 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:33.233394 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5df57f-9d4f-4bfb-bdb9-ec4e4bf30258 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:33.233562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5df57f-9d4f-4bfb-bdb9-ec4e4bf30258 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:33.235011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 83/160] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:33.235922 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fee25-e448-4e0d-8cb9-6d18572fd8b8 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:33.236248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fee25-e448-4e0d-8cb9-6d18572fd8b8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:33.236531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fee25-e448-4e0d-8cb9-6d18572fd8b8 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:33.238292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 84/161] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:33.238715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fee25-e448-4e0d-8cb9-6d18572fd8b8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:33.282448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7fc49-b562-43ac-9121-67003c92cd86 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:33.326421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a8418-6d22-401b-a44e-cf41aea56846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:33.327794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a8418-6d22-401b-a44e-cf41aea56846 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:33.327794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a8418-6d22-401b-a44e-cf41aea56846 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:33.327794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a8418-6d22-401b-a44e-cf41aea56846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:33.330226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a8418-6d22-401b-a44e-cf41aea56846 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=b81760932eec4beda4343070aaf9b956, user.id=819f4355d22549de99860edd3df36888) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:33.330226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a8418-6d22-401b-a44e-cf41aea56846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:33.331311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a8418-6d22-401b-a44e-cf41aea56846 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:33.338922 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 78/162] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:33 2022] GET /identity/v3/role_assignments?user.id=819f4355d22549de99860edd3df36888&scope.project.id=b81760932eec4beda4343070aaf9b956&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:33.340242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9829a58-60fa-4a2d-95da-6871129463df None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:33.340533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9829a58-60fa-4a2d-95da-6871129463df None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:33.340811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9829a58-60fa-4a2d-95da-6871129463df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:33.371534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d9829a58-60fa-4a2d-95da-6871129463df None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:33.372786 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d9829a58-60fa-4a2d-95da-6871129463df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:33.376725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fee25-e448-4e0d-8cb9-6d18572fd8b8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:33.402062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:33.417683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda4209e-2038-4d28-b33e-c4a2cdfeb4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:33.418842 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4209e-2038-4d28-b33e-c4a2cdfeb4a3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:33.419391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4209e-2038-4d28-b33e-c4a2cdfeb4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:33.419827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda4209e-2038-4d28-b33e-c4a2cdfeb4a3 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:33.420789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4209e-2038-4d28-b33e-c4a2cdfeb4a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:33.421300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4209e-2038-4d28-b33e-c4a2cdfeb4a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:33.424475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda4209e-2038-4d28-b33e-c4a2cdfeb4a3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:33.427530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:33.438503 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 85/163] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:33.439790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f183c-0aa1-48e3-8c99-12268914177e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:33.440127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f183c-0aa1-48e3-8c99-12268914177e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:33.440406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f183c-0aa1-48e3-8c99-12268914177e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:33.458999 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-533f183c-0aa1-48e3-8c99-12268914177e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:33.460173 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-533f183c-0aa1-48e3-8c99-12268914177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:33.466725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 79/164] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:33.467498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9829a58-60fa-4a2d-95da-6871129463df None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:33.559367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 86/165] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:33.560115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f183c-0aa1-48e3-8c99-12268914177e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:33.583363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9829a58-60fa-4a2d-95da-6871129463df None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:33.623858 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b9572-d4cf-4b12-a526-46a9b59d85dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:33.625090 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b9572-d4cf-4b12-a526-46a9b59d85dc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:33.625377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b9572-d4cf-4b12-a526-46a9b59d85dc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:33.625978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b9572-d4cf-4b12-a526-46a9b59d85dc None admin] PATH_INFO: `/v3/users/ea069f8de53148a798edcfd4984feff3` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:33.627387 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b9572-d4cf-4b12-a526-46a9b59d85dc None admin] RBAC: Authorizing `identity:get_user(user_id=ea069f8de53148a798edcfd4984feff3)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:33.629900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0b9572-d4cf-4b12-a526-46a9b59d85dc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:33.631672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 80/166] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:33 2022] GET /identity/v3/users/ea069f8de53148a798edcfd4984feff3 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:33.633426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b9572-d4cf-4b12-a526-46a9b59d85dc None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:33.665446 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:33.689805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f183c-0aa1-48e3-8c99-12268914177e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:33.724193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:33.739082 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64915380-9fba-4773-8b3a-175e8e896932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:33.740317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64915380-9fba-4773-8b3a-175e8e896932 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:33.740317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64915380-9fba-4773-8b3a-175e8e896932 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:33.740482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64915380-9fba-4773-8b3a-175e8e896932 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:33.753466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64915380-9fba-4773-8b3a-175e8e896932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-64915380-9fba-4773-8b3a-175e8e896932 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:33.762222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:33.763171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 87/167] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:33.765740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64915380-9fba-4773-8b3a-175e8e896932 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:33.781548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0b9572-d4cf-4b12-a526-46a9b59d85dc None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:33.822354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc58af1-bcd9-47c1-9cc4-5db6b862a64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:33.823231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc58af1-bcd9-47c1-9cc4-5db6b862a64d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:33.823448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc58af1-bcd9-47c1-9cc4-5db6b862a64d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:33.823722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc58af1-bcd9-47c1-9cc4-5db6b862a64d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:33.834552 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc58af1-bcd9-47c1-9cc4-5db6b862a64d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-7cc58af1-bcd9-47c1-9cc4-5db6b862a64d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:33.842440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:33.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:33.844766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 81/168] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:35:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:33.848709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc58af1-bcd9-47c1-9cc4-5db6b862a64d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:33.888570 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64915380-9fba-4773-8b3a-175e8e896932 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:33.930861 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb6d25-1738-47fb-817f-0f01e8ca93d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:33.932080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb6d25-1738-47fb-817f-0f01e8ca93d3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:33.932381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb6d25-1738-47fb-817f-0f01e8ca93d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:33.932659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb6d25-1738-47fb-817f-0f01e8ca93d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:33.945497 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:33.973313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc58af1-bcd9-47c1-9cc4-5db6b862a64d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:34.026620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:34.042066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb6d25-1738-47fb-817f-0f01e8ca93d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:34.043829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb6d25-1738-47fb-817f-0f01e8ca93d3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:34.051479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb467eb-32b3-485d-9d9c-2fbe8735be32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:34.052190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb467eb-32b3-485d-9d9c-2fbe8735be32 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:34.052479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb467eb-32b3-485d-9d9c-2fbe8735be32 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:34.052762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb467eb-32b3-485d-9d9c-2fbe8735be32 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:34.053520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb467eb-32b3-485d-9d9c-2fbe8735be32 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:34.053831 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb467eb-32b3-485d-9d9c-2fbe8735be32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:34.055295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb467eb-32b3-485d-9d9c-2fbe8735be32 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:34.073827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 82/169] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:34.075401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb467eb-32b3-485d-9d9c-2fbe8735be32 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:34.214003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb467eb-32b3-485d-9d9c-2fbe8735be32 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:34.252392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 88/170] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:34.253270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb6d25-1738-47fb-817f-0f01e8ca93d3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:34.264421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646068cc-74ac-4bf5-bb16-fb374afb4b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:34.265100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646068cc-74ac-4bf5-bb16-fb374afb4b72 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:34.265379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646068cc-74ac-4bf5-bb16-fb374afb4b72 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:34.265649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646068cc-74ac-4bf5-bb16-fb374afb4b72 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:34.266405 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646068cc-74ac-4bf5-bb16-fb374afb4b72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:34.266709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646068cc-74ac-4bf5-bb16-fb374afb4b72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:34.268118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646068cc-74ac-4bf5-bb16-fb374afb4b72 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:34.282755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 83/171] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:35:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:34.283819 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646068cc-74ac-4bf5-bb16-fb374afb4b72 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:34.311897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:34.425453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb6d25-1738-47fb-817f-0f01e8ca93d3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:34.448427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:34.460982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749d0ae-324f-42c2-8c9e-643e3448e5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:34.461880 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749d0ae-324f-42c2-8c9e-643e3448e5fc None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:34.462165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749d0ae-324f-42c2-8c9e-643e3448e5fc None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:34.462440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749d0ae-324f-42c2-8c9e-643e3448e5fc None admin] PATH_INFO: `/v3/users/4aa94cd636fc48ed938817335b7870ca` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:34.463666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749d0ae-324f-42c2-8c9e-643e3448e5fc None admin] RBAC: Authorizing `identity:get_user(user_id=4aa94cd636fc48ed938817335b7870ca)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:34.465239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d749d0ae-324f-42c2-8c9e-643e3448e5fc None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:34.467727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 89/172] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:34 2022] GET /identity/v3/users/4aa94cd636fc48ed938817335b7870ca => generated 289 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:34.468528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749d0ae-324f-42c2-8c9e-643e3448e5fc None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:34.521977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646068cc-74ac-4bf5-bb16-fb374afb4b72 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:34.576515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749d0ae-324f-42c2-8c9e-643e3448e5fc None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:34.612619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf52b8-6333-47e7-97a8-bc2c3f849ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:34.613741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf52b8-6333-47e7-97a8-bc2c3f849ce8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:34.614023 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf52b8-6333-47e7-97a8-bc2c3f849ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:34.614298 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf52b8-6333-47e7-97a8-bc2c3f849ce8 None admin] PATH_INFO: `/v3/users/ea069f8de53148a798edcfd4984feff3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:34.615525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf52b8-6333-47e7-97a8-bc2c3f849ce8 None admin] RBAC: Authorizing `identity:get_user(user_id=ea069f8de53148a798edcfd4984feff3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:34.617435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf52b8-6333-47e7-97a8-bc2c3f849ce8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:34.619232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 90/173] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:34 2022] GET /identity/v3/users/ea069f8de53148a798edcfd4984feff3 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:34.619838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf52b8-6333-47e7-97a8-bc2c3f849ce8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:34.636771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47219ef-6908-4344-944f-32c279faae1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:34.637880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47219ef-6908-4344-944f-32c279faae1a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:34.638171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47219ef-6908-4344-944f-32c279faae1a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:34.638454 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47219ef-6908-4344-944f-32c279faae1a None admin] PATH_INFO: `/v3/projects/e442b172ca2f41339963bba006c17769` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:34.645018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47219ef-6908-4344-944f-32c279faae1a None admin] RBAC: Authorizing `identity:get_project(project_id=e442b172ca2f41339963bba006c17769)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:34.646524 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47219ef-6908-4344-944f-32c279faae1a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:34.649494 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 84/174] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:34 2022] GET /identity/v3/projects/e442b172ca2f41339963bba006c17769 => generated 312 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:34.650299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47219ef-6908-4344-944f-32c279faae1a None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:34.660417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:34.729556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf52b8-6333-47e7-97a8-bc2c3f849ce8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:34.764862 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d283f66-15e6-4942-b35c-33911c55e8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:34.766084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283f66-15e6-4942-b35c-33911c55e8a3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:34.766371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283f66-15e6-4942-b35c-33911c55e8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:34.766652 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d283f66-15e6-4942-b35c-33911c55e8a3 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:34.768016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d283f66-15e6-4942-b35c-33911c55e8a3 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:34.769531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d283f66-15e6-4942-b35c-33911c55e8a3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:34.771457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 91/175] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:34 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:34.772206 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d283f66-15e6-4942-b35c-33911c55e8a3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:34.885570 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d283f66-15e6-4942-b35c-33911c55e8a3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:34.900356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:34.921984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de71413-b39d-4684-a5b7-0d6deb695769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:34.922646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de71413-b39d-4684-a5b7-0d6deb695769 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:34.922924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de71413-b39d-4684-a5b7-0d6deb695769 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:34.923196 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de71413-b39d-4684-a5b7-0d6deb695769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:34.924072 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de71413-b39d-4684-a5b7-0d6deb695769 None admin] RBAC: Adding query filter params (role.id=8f3ea95f038d4d8fbd1fd4897be36344, scope.project.id=e442b172ca2f41339963bba006c17769, user.id=ea069f8de53148a798edcfd4984feff3) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:34.924389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de71413-b39d-4684-a5b7-0d6deb695769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:34.925853 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de71413-b39d-4684-a5b7-0d6deb695769 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:34.932029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:34.933591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 92/176] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:34 2022] GET /identity/v3/role_assignments?user.id=ea069f8de53148a798edcfd4984feff3&scope.project.id=e442b172ca2f41339963bba006c17769&role.id=8f3ea95f038d4d8fbd1fd4897be36344 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:34.934926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de71413-b39d-4684-a5b7-0d6deb695769 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:35.006339 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47219ef-6908-4344-944f-32c279faae1a None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:35.110944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c221d08-9569-4385-a6ee-f89209cb5f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:35.111618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c221d08-9569-4385-a6ee-f89209cb5f71 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:35.111900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c221d08-9569-4385-a6ee-f89209cb5f71 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:35.112202 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c221d08-9569-4385-a6ee-f89209cb5f71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:35.128254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de71413-b39d-4684-a5b7-0d6deb695769 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:35.186719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ced15c-c4b6-4f94-a72a-0a230e7be908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:35.187551 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ced15c-c4b6-4f94-a72a-0a230e7be908 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:35.187834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ced15c-c4b6-4f94-a72a-0a230e7be908 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:35.188220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ced15c-c4b6-4f94-a72a-0a230e7be908 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956/users/4aa94cd636fc48ed938817335b7870ca/roles/f740696dd1a847929b6236dcca34cc2f` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:35.190239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ced15c-c4b6-4f94-a72a-0a230e7be908 None admin] RBAC: Authorizing `identity:create_grant(project_id=b81760932eec4beda4343070aaf9b956, user_id=4aa94cd636fc48ed938817335b7870ca, role_id=f740696dd1a847929b6236dcca34cc2f)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:35.191733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ced15c-c4b6-4f94-a72a-0a230e7be908 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:35.206909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 93/177] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:35:34 2022] PUT /identity/v3/projects/b81760932eec4beda4343070aaf9b956/users/4aa94cd636fc48ed938817335b7870ca/roles/f740696dd1a847929b6236dcca34cc2f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:35:35.224508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c221d08-9569-4385-a6ee-f89209cb5f71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:35.226085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c221d08-9569-4385-a6ee-f89209cb5f71 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:35.309238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:35.333271 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 85/178] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:35.338994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ced15c-c4b6-4f94-a72a-0a230e7be908 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:35.374355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:35.490148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ced15c-c4b6-4f94-a72a-0a230e7be908 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:35.530148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a044fe-1130-428b-8097-e58c014bffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:35.531122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a044fe-1130-428b-8097-e58c014bffbb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:35.531418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a044fe-1130-428b-8097-e58c014bffbb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:35.531733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a044fe-1130-428b-8097-e58c014bffbb None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:35.533373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a044fe-1130-428b-8097-e58c014bffbb None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:35.534903 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a044fe-1130-428b-8097-e58c014bffbb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:35.536979 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 94/179] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:35 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:35.543874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c221d08-9569-4385-a6ee-f89209cb5f71 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:35.686125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c221d08-9569-4385-a6ee-f89209cb5f71 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:35.734393 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5bfd9-0e8e-4d5c-9726-f7f7c2940755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:35.735364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5bfd9-0e8e-4d5c-9726-f7f7c2940755 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:35.735821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5bfd9-0e8e-4d5c-9726-f7f7c2940755 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:35.736340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5bfd9-0e8e-4d5c-9726-f7f7c2940755 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:35.737370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5bfd9-0e8e-4d5c-9726-f7f7c2940755 None admin] RBAC: Adding query filter params (role.id=ff6066b1fb8940cd9ec82c078fd8d2f6, scope.project.id=b81760932eec4beda4343070aaf9b956, user.id=ea069f8de53148a798edcfd4984feff3) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:35.737870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5bfd9-0e8e-4d5c-9726-f7f7c2940755 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:35.739547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5bfd9-0e8e-4d5c-9726-f7f7c2940755 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:35.746443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 86/180] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:35 2022] GET /identity/v3/role_assignments?user.id=ea069f8de53148a798edcfd4984feff3&scope.project.id=b81760932eec4beda4343070aaf9b956&role.id=ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:35.849899 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23684252-dcad-48f8-86e2-48fc8c200022 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:35.850323 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23684252-dcad-48f8-86e2-48fc8c200022 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:35.850573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23684252-dcad-48f8-86e2-48fc8c200022 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:35.852379 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 95/181] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:35.858458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87ae48-1128-49cc-ab10-a9529e839cc1 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:35.859106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87ae48-1128-49cc-ab10-a9529e839cc1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:35.859602 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87ae48-1128-49cc-ab10-a9529e839cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:35.878512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-3c87ae48-1128-49cc-ab10-a9529e839cc1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:35.880147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3c87ae48-1128-49cc-ab10-a9529e839cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:35.924098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:35.975053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 87/182] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:35.998795 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23684252-dcad-48f8-86e2-48fc8c200022 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:36.032277 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c114f67-af30-4b93-92b3-2da6ed3fdf98 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.033094 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c114f67-af30-4b93-92b3-2da6ed3fdf98 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.033164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c114f67-af30-4b93-92b3-2da6ed3fdf98 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.034651 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 88/183] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:35:36.039722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267977fb-7826-46d3-b228-1543cb80c458 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.040059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267977fb-7826-46d3-b228-1543cb80c458 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.040499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267977fb-7826-46d3-b228-1543cb80c458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.058057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-267977fb-7826-46d3-b228-1543cb80c458 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:36.059630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-267977fb-7826-46d3-b228-1543cb80c458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:36.131689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 89/184] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:36.154188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23684252-dcad-48f8-86e2-48fc8c200022 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:36.158244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:36.159148 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267977fb-7826-46d3-b228-1543cb80c458 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:36.198757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a3f33-1486-4972-a197-cfe06183c863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:36.199580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a3f33-1486-4972-a197-cfe06183c863 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.199857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a3f33-1486-4972-a197-cfe06183c863 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.200152 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a3f33-1486-4972-a197-cfe06183c863 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.218635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a3f33-1486-4972-a197-cfe06183c863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-2c8a3f33-1486-4972-a197-cfe06183c863 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:36.227428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:36.228380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:36.229174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 96/185] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:35:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:36.235465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a3f33-1486-4972-a197-cfe06183c863 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:36.297293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:36.301218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267977fb-7826-46d3-b228-1543cb80c458 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:36.340890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ac2b8-1a3e-49a2-8f2a-9dae56cadcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:36.341725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ac2b8-1a3e-49a2-8f2a-9dae56cadcdb None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.342018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ac2b8-1a3e-49a2-8f2a-9dae56cadcdb None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.342319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ac2b8-1a3e-49a2-8f2a-9dae56cadcdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.344895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a3f33-1486-4972-a197-cfe06183c863 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:36.355141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ac2b8-1a3e-49a2-8f2a-9dae56cadcdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-8a6ac2b8-1a3e-49a2-8f2a-9dae56cadcdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:36.362763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:36.363648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 90/186] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:35:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:36.365466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ed08d-3081-4b18-9981-87dfd0e23fc8 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.366190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ed08d-3081-4b18-9981-87dfd0e23fc8 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.366190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ed08d-3081-4b18-9981-87dfd0e23fc8 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.367114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 91/187] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:36.370116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ed08d-3081-4b18-9981-87dfd0e23fc8 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:36.387576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c124ad5-1710-409c-ba26-e2335e3dfde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:36.388463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c124ad5-1710-409c-ba26-e2335e3dfde3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.388762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c124ad5-1710-409c-ba26-e2335e3dfde3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.389048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c124ad5-1710-409c-ba26-e2335e3dfde3 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.389865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c124ad5-1710-409c-ba26-e2335e3dfde3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:36.390200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c124ad5-1710-409c-ba26-e2335e3dfde3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:36.391719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c124ad5-1710-409c-ba26-e2335e3dfde3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:36.400696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 97/188] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:35:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:36.401909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ff020-e470-42fb-b691-d5ba445295fc None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.402201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ff020-e470-42fb-b691-d5ba445295fc None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.402477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ff020-e470-42fb-b691-d5ba445295fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.415094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:36.421871 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-de1ff020-e470-42fb-b691-d5ba445295fc None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:36.423149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-de1ff020-e470-42fb-b691-d5ba445295fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:36.484953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ed08d-3081-4b18-9981-87dfd0e23fc8 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:36.491336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 98/189] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:36.501222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ff020-e470-42fb-b691-d5ba445295fc None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:36.520638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16559e7b-1f8b-4c0a-aaa5-8a79f6df1b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:36.521306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16559e7b-1f8b-4c0a-aaa5-8a79f6df1b17 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.521709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16559e7b-1f8b-4c0a-aaa5-8a79f6df1b17 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.521951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16559e7b-1f8b-4c0a-aaa5-8a79f6df1b17 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.522716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16559e7b-1f8b-4c0a-aaa5-8a79f6df1b17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:36.523030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16559e7b-1f8b-4c0a-aaa5-8a79f6df1b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:36.524460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16559e7b-1f8b-4c0a-aaa5-8a79f6df1b17 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:36.534368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 92/190] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:36.535025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16559e7b-1f8b-4c0a-aaa5-8a79f6df1b17 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:36.563209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:36.616191 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ff020-e470-42fb-b691-d5ba445295fc None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:36.647894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16559e7b-1f8b-4c0a-aaa5-8a79f6df1b17 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:36.651883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2876e43-deb5-4761-9603-acb1073bf4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:36.652829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2876e43-deb5-4761-9603-acb1073bf4e6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.653114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2876e43-deb5-4761-9603-acb1073bf4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.653387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2876e43-deb5-4761-9603-acb1073bf4e6 None admin] PATH_INFO: `/v3/users/e72bf81dbd93487897393b43a6e4fdc9` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.657558 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:36.669821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2876e43-deb5-4761-9603-acb1073bf4e6 None admin] RBAC: Authorizing `identity:get_user(user_id=e72bf81dbd93487897393b43a6e4fdc9)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:36.671312 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2876e43-deb5-4761-9603-acb1073bf4e6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:36.673271 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 99/191] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:36 2022] GET /identity/v3/users/e72bf81dbd93487897393b43a6e4fdc9 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:36.673856 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2876e43-deb5-4761-9603-acb1073bf4e6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:36.690978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bf908-6a30-4358-8641-07328b0224e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:36.692295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bf908-6a30-4358-8641-07328b0224e6 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.692645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bf908-6a30-4358-8641-07328b0224e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.692923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bf908-6a30-4358-8641-07328b0224e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.694154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bf908-6a30-4358-8641-07328b0224e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:36.695726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bf908-6a30-4358-8641-07328b0224e6 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:36.697548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 93/192] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:36.698232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bf908-6a30-4358-8641-07328b0224e6 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:36.779130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2876e43-deb5-4761-9603-acb1073bf4e6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:36.793009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:36.803749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bf908-6a30-4358-8641-07328b0224e6 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:36.814189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff8848-d107-4b08-835b-96c14fe96676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:36.815302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff8848-d107-4b08-835b-96c14fe96676 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.815580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff8848-d107-4b08-835b-96c14fe96676 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.815850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff8848-d107-4b08-835b-96c14fe96676 None admin] PATH_INFO: `/v3/users/e72bf81dbd93487897393b43a6e4fdc9` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.817074 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff8848-d107-4b08-835b-96c14fe96676 None admin] RBAC: Authorizing `identity:get_user(user_id=e72bf81dbd93487897393b43a6e4fdc9)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:36.818581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff8848-d107-4b08-835b-96c14fe96676 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:36.820251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 100/193] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:36 2022] GET /identity/v3/users/e72bf81dbd93487897393b43a6e4fdc9 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:36.824842 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff8848-d107-4b08-835b-96c14fe96676 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:36.847569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb5881-7046-457e-8a95-0509e30fc6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:36.848237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb5881-7046-457e-8a95-0509e30fc6a5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.848530 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb5881-7046-457e-8a95-0509e30fc6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.848784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb5881-7046-457e-8a95-0509e30fc6a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.886196 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:36.919180 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb5881-7046-457e-8a95-0509e30fc6a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:36.920751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb5881-7046-457e-8a95-0509e30fc6a5 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:36.934335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff8848-d107-4b08-835b-96c14fe96676 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:36.977433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bed4a-183f-410b-9600-50b04322f026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:36.978123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bed4a-183f-410b-9600-50b04322f026 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:36.978411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bed4a-183f-410b-9600-50b04322f026 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:36.978688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bed4a-183f-410b-9600-50b04322f026 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:36.979413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bed4a-183f-410b-9600-50b04322f026 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:36.980929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bed4a-183f-410b-9600-50b04322f026 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:37.002247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 101/194] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:35:36 2022] POST /identity/v3/users => generated 297 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:37.003457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bed4a-183f-410b-9600-50b04322f026 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:37.028967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:37.041011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 94/195] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:37.048227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb5881-7046-457e-8a95-0509e30fc6a5 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:37.152417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bed4a-183f-410b-9600-50b04322f026 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:37.158786 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb5881-7046-457e-8a95-0509e30fc6a5 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:37.171414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:37.194885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee504c24-21c7-4c9f-8004-8911dc0fd8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:37.195784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee504c24-21c7-4c9f-8004-8911dc0fd8ae None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:37.196273 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee504c24-21c7-4c9f-8004-8911dc0fd8ae None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:37.196796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee504c24-21c7-4c9f-8004-8911dc0fd8ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:37.203701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106b31e-742c-4543-bb1d-5df7ae683478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:37.204595 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106b31e-742c-4543-bb1d-5df7ae683478 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:37.204875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106b31e-742c-4543-bb1d-5df7ae683478 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:37.205157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106b31e-742c-4543-bb1d-5df7ae683478 None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:37.206503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106b31e-742c-4543-bb1d-5df7ae683478 None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:37.207954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106b31e-742c-4543-bb1d-5df7ae683478 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:37.210118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 95/196] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:37 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:37.216472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106b31e-742c-4543-bb1d-5df7ae683478 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:37.275060 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee504c24-21c7-4c9f-8004-8911dc0fd8ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:37.276907 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee504c24-21c7-4c9f-8004-8911dc0fd8ae None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:37.291228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:37.351359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106b31e-742c-4543-bb1d-5df7ae683478 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:37.396621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad425f-43e5-455a-9da7-c9345f4a2d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:37.397593 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad425f-43e5-455a-9da7-c9345f4a2d5e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:37.398086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad425f-43e5-455a-9da7-c9345f4a2d5e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:37.398556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad425f-43e5-455a-9da7-c9345f4a2d5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:37.399320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 102/197] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:37.399752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad425f-43e5-455a-9da7-c9345f4a2d5e None admin] RBAC: Adding query filter params (role.id=ff6066b1fb8940cd9ec82c078fd8d2f6, scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62, user.id=e72bf81dbd93487897393b43a6e4fdc9) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:37.400251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad425f-43e5-455a-9da7-c9345f4a2d5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:37.401896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad425f-43e5-455a-9da7-c9345f4a2d5e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:37.405361 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee504c24-21c7-4c9f-8004-8911dc0fd8ae None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:37.409891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 96/198] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:37 2022] GET /identity/v3/role_assignments?user.id=e72bf81dbd93487897393b43a6e4fdc9&scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62&role.id=ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:37.429515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:37.523090 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee504c24-21c7-4c9f-8004-8911dc0fd8ae None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:37.563083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2731b-b841-4907-8294-1065650bea0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:37.563994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2731b-b841-4907-8294-1065650bea0c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:37.564315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2731b-b841-4907-8294-1065650bea0c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:37.564884 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2731b-b841-4907-8294-1065650bea0c None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:37.566200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2731b-b841-4907-8294-1065650bea0c None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:37.567739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2731b-b841-4907-8294-1065650bea0c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:37.569813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 103/199] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:37 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:37.575896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad425f-43e5-455a-9da7-c9345f4a2d5e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:37.614245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f314337-dfb9-4b6b-a008-30fa55ebe65e None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:37.614559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f314337-dfb9-4b6b-a008-30fa55ebe65e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:37.615021 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f314337-dfb9-4b6b-a008-30fa55ebe65e None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:37.616926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 104/200] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:35:37.622429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef34249-fd5a-4d80-8a1f-d848881af069 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:37.622716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef34249-fd5a-4d80-8a1f-d848881af069 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:37.622997 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef34249-fd5a-4d80-8a1f-d848881af069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:37.641048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1ef34249-fd5a-4d80-8a1f-d848881af069 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:37.642249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1ef34249-fd5a-4d80-8a1f-d848881af069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:37.714853 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 105/201] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:37.738788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef34249-fd5a-4d80-8a1f-d848881af069 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:37.739564 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad425f-43e5-455a-9da7-c9345f4a2d5e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:37.768430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:37.783641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:37.808024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd883c02-21ec-4eae-ab6b-77ca994bd0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:37.808759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd883c02-21ec-4eae-ab6b-77ca994bd0a6 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:37.809048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd883c02-21ec-4eae-ab6b-77ca994bd0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:37.809321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd883c02-21ec-4eae-ab6b-77ca994bd0a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:37.810186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd883c02-21ec-4eae-ab6b-77ca994bd0a6 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62, user.id=e72bf81dbd93487897393b43a6e4fdc9) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:37.810499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd883c02-21ec-4eae-ab6b-77ca994bd0a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:37.811994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd883c02-21ec-4eae-ab6b-77ca994bd0a6 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:37.825862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 97/202] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:37 2022] GET /identity/v3/role_assignments?user.id=e72bf81dbd93487897393b43a6e4fdc9&scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:37.895119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef34249-fd5a-4d80-8a1f-d848881af069 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:37.954416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc90d97-d1b3-4244-9769-0dc2effdabce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:37.955329 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc90d97-d1b3-4244-9769-0dc2effdabce None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:37.955611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc90d97-d1b3-4244-9769-0dc2effdabce None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:37.955934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc90d97-d1b3-4244-9769-0dc2effdabce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:37.971440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc90d97-d1b3-4244-9769-0dc2effdabce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-8dc90d97-d1b3-4244-9769-0dc2effdabce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:37.980110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:37.981313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:37.982279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:37.982279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:37.982279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:37.982279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:37.982279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:37.982279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:37.982279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:37.982279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:37.982279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:37.982279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:37.982279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:37.982279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:37.984837 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 106/203] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:35:37.991135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd883c02-21ec-4eae-ab6b-77ca994bd0a6 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:38.119670 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd883c02-21ec-4eae-ab6b-77ca994bd0a6 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:38.161899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c91e02-74eb-4edd-ad0c-728785bd1207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:38.162812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c91e02-74eb-4edd-ad0c-728785bd1207 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:38.163235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c91e02-74eb-4edd-ad0c-728785bd1207 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:38.163648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c91e02-74eb-4edd-ad0c-728785bd1207 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:38.164566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c91e02-74eb-4edd-ad0c-728785bd1207 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:38.165033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c91e02-74eb-4edd-ad0c-728785bd1207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:38.166559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c91e02-74eb-4edd-ad0c-728785bd1207 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:38.175273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 98/204] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:38.180961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc90d97-d1b3-4244-9769-0dc2effdabce None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:38.275174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:38.307652 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc90d97-d1b3-4244-9769-0dc2effdabce None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:38.343311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aadc05-143b-41a1-9a9f-9874f4b42021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:38.344237 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aadc05-143b-41a1-9a9f-9874f4b42021 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:38.344519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aadc05-143b-41a1-9a9f-9874f4b42021 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:38.344787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aadc05-143b-41a1-9a9f-9874f4b42021 None admin] PATH_INFO: `/v3/users/4aa94cd636fc48ed938817335b7870ca` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:38.345990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aadc05-143b-41a1-9a9f-9874f4b42021 None admin] RBAC: Authorizing `identity:get_user(user_id=4aa94cd636fc48ed938817335b7870ca)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:38.347438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aadc05-143b-41a1-9a9f-9874f4b42021 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:38.349447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 107/205] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:38 2022] GET /identity/v3/users/4aa94cd636fc48ed938817335b7870ca => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:38.354822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c91e02-74eb-4edd-ad0c-728785bd1207 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:38.480399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:38.546833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c91e02-74eb-4edd-ad0c-728785bd1207 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:38.594990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462eae9e-4725-49ec-a5ae-c2a80c18f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:38.595647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462eae9e-4725-49ec-a5ae-c2a80c18f385 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:38.595931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462eae9e-4725-49ec-a5ae-c2a80c18f385 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:38.596233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462eae9e-4725-49ec-a5ae-c2a80c18f385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:38.686259 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462eae9e-4725-49ec-a5ae-c2a80c18f385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:38.687754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462eae9e-4725-49ec-a5ae-c2a80c18f385 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:38.749085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:38.820853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 99/206] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:38.826781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aadc05-143b-41a1-9a9f-9874f4b42021 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:38.997286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aadc05-143b-41a1-9a9f-9874f4b42021 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:39.039045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcc1cd-f583-467d-ab7e-486daf3a1a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:39.040310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcc1cd-f583-467d-ab7e-486daf3a1a69 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:39.040784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcc1cd-f583-467d-ab7e-486daf3a1a69 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:39.041244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcc1cd-f583-467d-ab7e-486daf3a1a69 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:39.042645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcc1cd-f583-467d-ab7e-486daf3a1a69 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:39.044313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcc1cd-f583-467d-ab7e-486daf3a1a69 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:39.046515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 108/207] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:38 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:39.057232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462eae9e-4725-49ec-a5ae-c2a80c18f385 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:39.207478 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462eae9e-4725-49ec-a5ae-c2a80c18f385 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:39.242436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6f917-87b3-4bd3-b99c-95d26533d71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:39.243405 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6f917-87b3-4bd3-b99c-95d26533d71c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:39.243903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6f917-87b3-4bd3-b99c-95d26533d71c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:39.244396 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6f917-87b3-4bd3-b99c-95d26533d71c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:39.245414 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6f917-87b3-4bd3-b99c-95d26533d71c None admin] RBAC: Adding query filter params (role.id=f740696dd1a847929b6236dcca34cc2f, scope.project.id=b81760932eec4beda4343070aaf9b956, user.id=4aa94cd636fc48ed938817335b7870ca) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:39.245934 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6f917-87b3-4bd3-b99c-95d26533d71c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:39.247572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6f917-87b3-4bd3-b99c-95d26533d71c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:39.253641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 100/208] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:39 2022] GET /identity/v3/role_assignments?user.id=4aa94cd636fc48ed938817335b7870ca&scope.project.id=b81760932eec4beda4343070aaf9b956&role.id=f740696dd1a847929b6236dcca34cc2f => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:39.595198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97640d3-0bb9-4a42-8bd2-49202a344b79 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:39.596091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97640d3-0bb9-4a42-8bd2-49202a344b79 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:39.596533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97640d3-0bb9-4a42-8bd2-49202a344b79 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:39.598328 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 109/209] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:39.603861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67056f1b-5409-4cf9-9e78-0556d470bc11 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:39.604438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67056f1b-5409-4cf9-9e78-0556d470bc11 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:39.604904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67056f1b-5409-4cf9-9e78-0556d470bc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:39.622526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-67056f1b-5409-4cf9-9e78-0556d470bc11 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:39.624056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-67056f1b-5409-4cf9-9e78-0556d470bc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:39.633772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:39.714667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 101/210] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:39.738171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97640d3-0bb9-4a42-8bd2-49202a344b79 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:39.754255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:39.859939 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97640d3-0bb9-4a42-8bd2-49202a344b79 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:39.900687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215db914-6161-4fa0-b143-93ad221c78f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:39.901849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215db914-6161-4fa0-b143-93ad221c78f1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:39.902349 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215db914-6161-4fa0-b143-93ad221c78f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:39.902816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215db914-6161-4fa0-b143-93ad221c78f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:39.914327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215db914-6161-4fa0-b143-93ad221c78f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-215db914-6161-4fa0-b143-93ad221c78f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:39.925126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:39.926113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:39.926921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:39.926921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:39.926921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:39.926921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:39.926921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:39.926921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:39.926921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:39.926921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:39.926921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:39.926921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:39.926921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:39.926921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:39.929766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 110/211] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:39.935215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67056f1b-5409-4cf9-9e78-0556d470bc11 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:40.028031 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb365dc-c5ba-4376-a439-54ade5840669 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.028545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb365dc-c5ba-4376-a439-54ade5840669 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.029339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb365dc-c5ba-4376-a439-54ade5840669 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.031950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 111/212] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:40.037236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77106fd-5747-46ec-89e6-b8986fee0586 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.037614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77106fd-5747-46ec-89e6-b8986fee0586 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.037769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77106fd-5747-46ec-89e6-b8986fee0586 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.039417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 112/213] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:40.040403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f2b37-c773-4d2d-a7d1-02055efa807f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.040882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f2b37-c773-4d2d-a7d1-02055efa807f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.041168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f2b37-c773-4d2d-a7d1-02055efa807f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.045112 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67056f1b-5409-4cf9-9e78-0556d470bc11 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:40.059936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-f91f2b37-c773-4d2d-a7d1-02055efa807f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:40.061221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f91f2b37-c773-4d2d-a7d1-02055efa807f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:40.081221 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03302a4-1d21-4332-9b00-e931aceb93f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:40.081892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03302a4-1d21-4332-9b00-e931aceb93f7 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.082232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03302a4-1d21-4332-9b00-e931aceb93f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.082510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03302a4-1d21-4332-9b00-e931aceb93f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.083258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03302a4-1d21-4332-9b00-e931aceb93f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:40.083564 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03302a4-1d21-4332-9b00-e931aceb93f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:40.084981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03302a4-1d21-4332-9b00-e931aceb93f7 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:40.093739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 102/214] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:40.094830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7c971-a5d7-4480-a98a-a84f7a716fb4 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.095119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7c971-a5d7-4480-a98a-a84f7a716fb4 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.095395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7c971-a5d7-4480-a98a-a84f7a716fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.101269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:40.111869 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8aa7c971-a5d7-4480-a98a-a84f7a716fb4 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:40.113068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8aa7c971-a5d7-4480-a98a-a84f7a716fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:40.135435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 113/215] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:40.136800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b49bb-0035-4fea-a085-e37ffabde251 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.137085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b49bb-0035-4fea-a085-e37ffabde251 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.137352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b49bb-0035-4fea-a085-e37ffabde251 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.138582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 114/216] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:40.142863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b49bb-0035-4fea-a085-e37ffabde251 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:40.142863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:40.188033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 103/217] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:40.188033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3b715-b207-4eae-928e-1d6afa225747 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.188033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3b715-b207-4eae-928e-1d6afa225747 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.188033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3b715-b207-4eae-928e-1d6afa225747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.203874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f5a3b715-b207-4eae-928e-1d6afa225747 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:40.205126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f5a3b715-b207-4eae-928e-1d6afa225747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:40.246487 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b49bb-0035-4fea-a085-e37ffabde251 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:40.270073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 104/218] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:40.270789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3b715-b207-4eae-928e-1d6afa225747 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:40.288226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac7461-bfc5-45af-aa3a-313339276c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:40.289452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac7461-bfc5-45af-aa3a-313339276c22 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.289731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac7461-bfc5-45af-aa3a-313339276c22 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.289999 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac7461-bfc5-45af-aa3a-313339276c22 None admin] PATH_INFO: `/v3/users/39d69e7aa10c4ac6a939422cb1abf944` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.303645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac7461-bfc5-45af-aa3a-313339276c22 None admin] RBAC: Authorizing `identity:get_user(user_id=39d69e7aa10c4ac6a939422cb1abf944)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:40.305161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac7461-bfc5-45af-aa3a-313339276c22 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:40.306850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 115/219] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:40 2022] GET /identity/v3/users/39d69e7aa10c4ac6a939422cb1abf944 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:40.307602 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac7461-bfc5-45af-aa3a-313339276c22 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:40.339048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:40.365621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:40.377200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3b715-b207-4eae-928e-1d6afa225747 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:40.406273 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac7461-bfc5-45af-aa3a-313339276c22 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:40.410775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7264317-021f-4cd7-9ba4-2ede95fab3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:40.411587 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7264317-021f-4cd7-9ba4-2ede95fab3b4 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.411870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7264317-021f-4cd7-9ba4-2ede95fab3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.412178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7264317-021f-4cd7-9ba4-2ede95fab3b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.422066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7264317-021f-4cd7-9ba4-2ede95fab3b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-b7264317-021f-4cd7-9ba4-2ede95fab3b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:40.429977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:40.430876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:40.431711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 105/220] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:35:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:40.432390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7264317-021f-4cd7-9ba4-2ede95fab3b4 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:40.440933 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f3e25-c527-444e-86a7-ff61f867d506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:40.441851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f3e25-c527-444e-86a7-ff61f867d506 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.442132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f3e25-c527-444e-86a7-ff61f867d506 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.442404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f3e25-c527-444e-86a7-ff61f867d506 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.443666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f3e25-c527-444e-86a7-ff61f867d506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:40.445183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f3e25-c527-444e-86a7-ff61f867d506 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:40.446801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 116/221] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:40.447550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f3e25-c527-444e-86a7-ff61f867d506 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:40.533779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7264317-021f-4cd7-9ba4-2ede95fab3b4 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:40.553475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f3e25-c527-444e-86a7-ff61f867d506 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:40.557013 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:40.569309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044a672-a90e-4f16-9be5-9342ce268b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:40.570101 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044a672-a90e-4f16-9be5-9342ce268b06 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.570380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044a672-a90e-4f16-9be5-9342ce268b06 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.570651 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044a672-a90e-4f16-9be5-9342ce268b06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.579093 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:40.580757 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044a672-a90e-4f16-9be5-9342ce268b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-e044a672-a90e-4f16-9be5-9342ce268b06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:40.588361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:40.589457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:40.595247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:40.595247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:40.595247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:40.595247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:40.595247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:40.595247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:40.595247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:40.595247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:40.595247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:40.595247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:40.595247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:40.595247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:40.595835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1c526-8e33-451a-8582-54fbecf1d51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:40.595835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1c526-8e33-451a-8582-54fbecf1d51a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.595835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1c526-8e33-451a-8582-54fbecf1d51a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.596382 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 106/222] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:35:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:40.596464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1c526-8e33-451a-8582-54fbecf1d51a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.596541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044a672-a90e-4f16-9be5-9342ce268b06 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:40.658392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1c526-8e33-451a-8582-54fbecf1d51a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:40.659799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1c526-8e33-451a-8582-54fbecf1d51a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:40.690926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044a672-a90e-4f16-9be5-9342ce268b06 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:40.723087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c85b2-0e4d-4aa3-8d52-4e4213857420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:40.723734 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c85b2-0e4d-4aa3-8d52-4e4213857420 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.724075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c85b2-0e4d-4aa3-8d52-4e4213857420 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.724354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c85b2-0e4d-4aa3-8d52-4e4213857420 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.725111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c85b2-0e4d-4aa3-8d52-4e4213857420 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:40.725408 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c85b2-0e4d-4aa3-8d52-4e4213857420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:40.726818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c85b2-0e4d-4aa3-8d52-4e4213857420 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:40.734899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 107/223] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:35:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:40.735680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c85b2-0e4d-4aa3-8d52-4e4213857420 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:40.758328 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 117/224] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:40.762076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1c526-8e33-451a-8582-54fbecf1d51a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:40.780156 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:40.789267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:40.832356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c85b2-0e4d-4aa3-8d52-4e4213857420 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:40.859875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1c526-8e33-451a-8582-54fbecf1d51a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:40.866611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997f33a-4879-4aa4-9cbb-35982bd36ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:40.867285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997f33a-4879-4aa4-9cbb-35982bd36ff0 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.867570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997f33a-4879-4aa4-9cbb-35982bd36ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.867844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997f33a-4879-4aa4-9cbb-35982bd36ff0 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.868578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997f33a-4879-4aa4-9cbb-35982bd36ff0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:40.870034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997f33a-4879-4aa4-9cbb-35982bd36ff0 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:40.893564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023c21e-5767-4f8f-8c1a-db191c37b518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:40.894241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023c21e-5767-4f8f-8c1a-db191c37b518 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:40.894526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023c21e-5767-4f8f-8c1a-db191c37b518 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:40.894801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023c21e-5767-4f8f-8c1a-db191c37b518 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:40.895089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 108/225] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:35:40 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:40.895543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023c21e-5767-4f8f-8c1a-db191c37b518 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:40.895848 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023c21e-5767-4f8f-8c1a-db191c37b518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:40.895941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997f33a-4879-4aa4-9cbb-35982bd36ff0 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:40.897271 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023c21e-5767-4f8f-8c1a-db191c37b518 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:40.906829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 118/226] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:40.907491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023c21e-5767-4f8f-8c1a-db191c37b518 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:41.019252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:41.028544 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997f33a-4879-4aa4-9cbb-35982bd36ff0 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:41.030170 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023c21e-5767-4f8f-8c1a-db191c37b518 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:41.030748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:41.074213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568e808-440b-4d64-9b3d-aec4fc615b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:41.075377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568e808-440b-4d64-9b3d-aec4fc615b1b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:41.075831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568e808-440b-4d64-9b3d-aec4fc615b1b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:41.076298 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568e808-440b-4d64-9b3d-aec4fc615b1b None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:41.077787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568e808-440b-4d64-9b3d-aec4fc615b1b None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:41.079636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568e808-440b-4d64-9b3d-aec4fc615b1b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:41.080705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09de399-a9fb-4408-a55c-cd0cc23ece7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:41.081917 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09de399-a9fb-4408-a55c-cd0cc23ece7b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:41.082022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 119/227] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:40 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:41.082278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09de399-a9fb-4408-a55c-cd0cc23ece7b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:41.082623 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09de399-a9fb-4408-a55c-cd0cc23ece7b None admin] PATH_INFO: `/v3/users/e72bf81dbd93487897393b43a6e4fdc9` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:41.083387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568e808-440b-4d64-9b3d-aec4fc615b1b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:41.085643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09de399-a9fb-4408-a55c-cd0cc23ece7b None admin] RBAC: Authorizing `identity:get_user(user_id=e72bf81dbd93487897393b43a6e4fdc9)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:41.087176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09de399-a9fb-4408-a55c-cd0cc23ece7b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:41.089849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 109/228] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:40 2022] GET /identity/v3/users/e72bf81dbd93487897393b43a6e4fdc9 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:41.090441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09de399-a9fb-4408-a55c-cd0cc23ece7b None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:41.236171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568e808-440b-4d64-9b3d-aec4fc615b1b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:41.243867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09de399-a9fb-4408-a55c-cd0cc23ece7b None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:41.287688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ec314-0d21-428c-a405-483492ada2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:41.289994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64adf36-e217-4d50-9a70-4ba930ed9e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:41.290452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ec314-0d21-428c-a405-483492ada2ba None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:41.290646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ec314-0d21-428c-a405-483492ada2ba None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:41.290923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ec314-0d21-428c-a405-483492ada2ba None admin] PATH_INFO: `/v3/users/e72bf81dbd93487897393b43a6e4fdc9` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:41.291719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64adf36-e217-4d50-9a70-4ba930ed9e9b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:41.292254 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64adf36-e217-4d50-9a70-4ba930ed9e9b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:41.292736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64adf36-e217-4d50-9a70-4ba930ed9e9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:41.293785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64adf36-e217-4d50-9a70-4ba930ed9e9b None admin] RBAC: Adding query filter params (role.id=8f3ea95f038d4d8fbd1fd4897be36344, scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62, user.id=39d69e7aa10c4ac6a939422cb1abf944) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:41.294315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64adf36-e217-4d50-9a70-4ba930ed9e9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:41.295098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ec314-0d21-428c-a405-483492ada2ba None admin] RBAC: Authorizing `identity:get_user(user_id=e72bf81dbd93487897393b43a6e4fdc9)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:41.296381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64adf36-e217-4d50-9a70-4ba930ed9e9b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:41.296626 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ec314-0d21-428c-a405-483492ada2ba None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:41.298544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 120/229] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:41 2022] GET /identity/v3/users/e72bf81dbd93487897393b43a6e4fdc9 => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:41.299114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ec314-0d21-428c-a405-483492ada2ba None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:41.304727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 110/230] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:41 2022] GET /identity/v3/role_assignments?user.id=39d69e7aa10c4ac6a939422cb1abf944&scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62&role.id=8f3ea95f038d4d8fbd1fd4897be36344 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:41.306053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64adf36-e217-4d50-9a70-4ba930ed9e9b None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:41.348353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:41.356934 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:41.427255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ec314-0d21-428c-a405-483492ada2ba None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:41.440325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64adf36-e217-4d50-9a70-4ba930ed9e9b None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:41.491665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f414a05-5c53-49c7-a16b-486ed46a1bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:41.493251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f414a05-5c53-49c7-a16b-486ed46a1bd3 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:41.493846 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f414a05-5c53-49c7-a16b-486ed46a1bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:41.494322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f414a05-5c53-49c7-a16b-486ed46a1bd3 None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:41.495937 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f414a05-5c53-49c7-a16b-486ed46a1bd3 None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:41.499075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f414a05-5c53-49c7-a16b-486ed46a1bd3 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:41.507839 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 111/231] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:41 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:41.514535 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f414a05-5c53-49c7-a16b-486ed46a1bd3 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:41.542663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee71fdc-d2c1-4f4f-911d-7d3ac3c87b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:41.544678 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee71fdc-d2c1-4f4f-911d-7d3ac3c87b7f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:41.545211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee71fdc-d2c1-4f4f-911d-7d3ac3c87b7f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:41.545680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee71fdc-d2c1-4f4f-911d-7d3ac3c87b7f None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:41.547425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee71fdc-d2c1-4f4f-911d-7d3ac3c87b7f None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:41.549259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee71fdc-d2c1-4f4f-911d-7d3ac3c87b7f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:41.553074 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 121/232] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:41 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:41.558788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee71fdc-d2c1-4f4f-911d-7d3ac3c87b7f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:41.734347 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:41.747327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f414a05-5c53-49c7-a16b-486ed46a1bd3 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:41.749382 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee71fdc-d2c1-4f4f-911d-7d3ac3c87b7f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:41.752681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:41.819503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c72497-4df9-4ce8-ace3-172811840157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:41.820440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c72497-4df9-4ce8-ace3-172811840157 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:41.820725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c72497-4df9-4ce8-ace3-172811840157 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:41.821003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c72497-4df9-4ce8-ace3-172811840157 None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62/users/e72bf81dbd93487897393b43a6e4fdc9/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:41.822426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9461cd-0e57-4b57-99ac-f3b434762c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:41.823327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9461cd-0e57-4b57-99ac-f3b434762c1b None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:41.823604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9461cd-0e57-4b57-99ac-f3b434762c1b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:41.823875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9461cd-0e57-4b57-99ac-f3b434762c1b None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62/users/e72bf81dbd93487897393b43a6e4fdc9/roles/ff6066b1fb8940cd9ec82c078fd8d2f6` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:41.826114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c72497-4df9-4ce8-ace3-172811840157 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bcbe8d7aaa3411b844ce5c42db14e62, user_id=e72bf81dbd93487897393b43a6e4fdc9, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:41.827581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c72497-4df9-4ce8-ace3-172811840157 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:41.831186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9461cd-0e57-4b57-99ac-f3b434762c1b None admin] RBAC: Authorizing `identity:create_grant(project_id=3bcbe8d7aaa3411b844ce5c42db14e62, user_id=e72bf81dbd93487897393b43a6e4fdc9, role_id=ff6066b1fb8940cd9ec82c078fd8d2f6)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:41.832671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9461cd-0e57-4b57-99ac-f3b434762c1b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:41.854664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 112/233] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:35:41 2022] PUT /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62/users/e72bf81dbd93487897393b43a6e4fdc9/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:35:41.856314 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 122/234] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:35:41 2022] PUT /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62/users/e72bf81dbd93487897393b43a6e4fdc9/roles/ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:35:43.564342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6047e-b888-4592-b1e0-e511a62f6365 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:43.564908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6047e-b888-4592-b1e0-e511a62f6365 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:43.564908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6047e-b888-4592-b1e0-e511a62f6365 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:43.566615 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 113/235] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:43.571318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d838eea-ec67-4201-bc4e-452bd1f5a778 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:43.571730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d838eea-ec67-4201-bc4e-452bd1f5a778 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:43.571911 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d838eea-ec67-4201-bc4e-452bd1f5a778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:43.585542 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7a2a6-bec2-4e70-a5d4-07c41e7c8106 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:43.585823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7a2a6-bec2-4e70-a5d4-07c41e7c8106 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:43.586095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7a2a6-bec2-4e70-a5d4-07c41e7c8106 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:43.587584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 114/236] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:43.591820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1d838eea-ec67-4201-bc4e-452bd1f5a778 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:43.592323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6131a6-18e0-47c9-b73c-fdc18704e993 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:43.592614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6131a6-18e0-47c9-b73c-fdc18704e993 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:43.592896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6131a6-18e0-47c9-b73c-fdc18704e993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:43.593088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1d838eea-ec67-4201-bc4e-452bd1f5a778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:43.628598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-5f6131a6-18e0-47c9-b73c-fdc18704e993 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:43.629807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5f6131a6-18e0-47c9-b73c-fdc18704e993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:43.697045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 123/237] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:43.718440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b974b4-e09c-4dda-bdf5-48f210f097e2 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:43.719047 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b974b4-e09c-4dda-bdf5-48f210f097e2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:43.719586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b974b4-e09c-4dda-bdf5-48f210f097e2 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:43.721439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 124/238] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:35:43.726428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f32c2-2e65-4711-86ff-b8d23bd51a7d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:43.726940 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f32c2-2e65-4711-86ff-b8d23bd51a7d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:43.727433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f32c2-2e65-4711-86ff-b8d23bd51a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:43.727768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 115/239] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:43.728699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6131a6-18e0-47c9-b73c-fdc18704e993 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:43.745624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7c3f32c2-2e65-4711-86ff-b8d23bd51a7d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:43.747257 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7c3f32c2-2e65-4711-86ff-b8d23bd51a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:43.828424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 125/240] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:43.829544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f32c2-2e65-4711-86ff-b8d23bd51a7d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:43.841321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:43.858205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:43.863262 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6131a6-18e0-47c9-b73c-fdc18704e993 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:43.906001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fa637-4936-42ac-879b-b2e1406fae29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:43.907373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fa637-4936-42ac-879b-b2e1406fae29 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:43.907909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fa637-4936-42ac-879b-b2e1406fae29 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:43.908512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fa637-4936-42ac-879b-b2e1406fae29 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:43.920061 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fa637-4936-42ac-879b-b2e1406fae29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-b21fa637-4936-42ac-879b-b2e1406fae29 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:43.928650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:43.929546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:43.930331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:43.930331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:43.930331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:43.930331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:43.930331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:43.930331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:43.930331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:43.930331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:43.930331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:43.930331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:43.930331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:43.930331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:43.932823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 116/241] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:43.933880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fa637-4936-42ac-879b-b2e1406fae29 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:43.950780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f32c2-2e65-4711-86ff-b8d23bd51a7d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:43.997436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89935a6-2cc9-46d3-a405-9a3f9ae2dd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:43.998396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89935a6-2cc9-46d3-a405-9a3f9ae2dd66 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:43.998690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89935a6-2cc9-46d3-a405-9a3f9ae2dd66 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:43.998959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89935a6-2cc9-46d3-a405-9a3f9ae2dd66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.000182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89935a6-2cc9-46d3-a405-9a3f9ae2dd66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:44.002506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89935a6-2cc9-46d3-a405-9a3f9ae2dd66 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:44.004836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 126/242] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:44.005834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89935a6-2cc9-46d3-a405-9a3f9ae2dd66 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:44.056176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fa637-4936-42ac-879b-b2e1406fae29 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:44.094833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db08da-18fc-4c75-acad-6a271262d31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:44.096151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db08da-18fc-4c75-acad-6a271262d31e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:44.096635 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db08da-18fc-4c75-acad-6a271262d31e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:44.097105 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db08da-18fc-4c75-acad-6a271262d31e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.108099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db08da-18fc-4c75-acad-6a271262d31e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:44.115347 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-43db08da-18fc-4c75-acad-6a271262d31e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:44.116802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:44.118245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:44.119027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:44.119027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:44.119027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:44.119027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:44.119027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:44.119027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:44.119027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:44.119027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:44.119027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:44.119027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:44.119027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:44.119027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:44.121715 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 117/243] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:44.122739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db08da-18fc-4c75-acad-6a271262d31e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:44.141067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:44.147095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89935a6-2cc9-46d3-a405-9a3f9ae2dd66 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:44.191002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb6b15-9b3c-4048-923d-b313b9561ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:44.191787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb6b15-9b3c-4048-923d-b313b9561ba9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:44.192115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb6b15-9b3c-4048-923d-b313b9561ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:44.192391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb6b15-9b3c-4048-923d-b313b9561ba9 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.193305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb6b15-9b3c-4048-923d-b313b9561ba9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:44.193616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb6b15-9b3c-4048-923d-b313b9561ba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:44.195619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb6b15-9b3c-4048-923d-b313b9561ba9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:44.206563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 127/244] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:44.207565 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb6b15-9b3c-4048-923d-b313b9561ba9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:44.244191 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db08da-18fc-4c75-acad-6a271262d31e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:44.285798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e69e47-934a-4370-8f24-014c9f5d8517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:44.286477 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e69e47-934a-4370-8f24-014c9f5d8517 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:44.286767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e69e47-934a-4370-8f24-014c9f5d8517 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:44.287047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e69e47-934a-4370-8f24-014c9f5d8517 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.287761 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e69e47-934a-4370-8f24-014c9f5d8517 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:44.289300 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e69e47-934a-4370-8f24-014c9f5d8517 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:44.313263 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 118/245] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:35:44 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:44.314530 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e69e47-934a-4370-8f24-014c9f5d8517 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:44.317327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb6b15-9b3c-4048-923d-b313b9561ba9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:44.358593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c99b93-6ae5-401f-b11c-725a65b667cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:44.359314 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c99b93-6ae5-401f-b11c-725a65b667cc None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:44.359601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c99b93-6ae5-401f-b11c-725a65b667cc None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:44.359876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c99b93-6ae5-401f-b11c-725a65b667cc None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.360743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c99b93-6ae5-401f-b11c-725a65b667cc None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:44.361052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c99b93-6ae5-401f-b11c-725a65b667cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:44.362556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c99b93-6ae5-401f-b11c-725a65b667cc None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:44.372410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 128/246] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:44.372410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cd1a0-34cd-495d-8c1c-34a13f83bf4a None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:44.372410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cd1a0-34cd-495d-8c1c-34a13f83bf4a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:44.372410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cd1a0-34cd-495d-8c1c-34a13f83bf4a None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.380172 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 129/247] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:44.381700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cd1a0-34cd-495d-8c1c-34a13f83bf4a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:44.399049 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:44.415035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:44.456859 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e69e47-934a-4370-8f24-014c9f5d8517 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:44.493224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52242c9-e8b8-4546-902a-08488c5a44c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:44.493762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52242c9-e8b8-4546-902a-08488c5a44c8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:44.494048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52242c9-e8b8-4546-902a-08488c5a44c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:44.494325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52242c9-e8b8-4546-902a-08488c5a44c8 None admin] PATH_INFO: `/v3/users/9eebec6e44714f71bdd7a774873a6978` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.513682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52242c9-e8b8-4546-902a-08488c5a44c8 None admin] RBAC: Authorizing `identity:get_user(user_id=9eebec6e44714f71bdd7a774873a6978)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:44.514906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cd1a0-34cd-495d-8c1c-34a13f83bf4a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:44.515200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52242c9-e8b8-4546-902a-08488c5a44c8 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:44.516995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 119/248] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:44 2022] GET /identity/v3/users/9eebec6e44714f71bdd7a774873a6978 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:44.520210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8305e51-d58e-4409-a89d-aa40ff57fcf2 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:44.520210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8305e51-d58e-4409-a89d-aa40ff57fcf2 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:44.520210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8305e51-d58e-4409-a89d-aa40ff57fcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.536814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-e8305e51-d58e-4409-a89d-aa40ff57fcf2 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:44.538017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e8305e51-d58e-4409-a89d-aa40ff57fcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:44.553034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf6ecb-a7a8-41fa-ae8c-00cd1236896f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:44.554936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf6ecb-a7a8-41fa-ae8c-00cd1236896f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:44.555222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf6ecb-a7a8-41fa-ae8c-00cd1236896f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:44.555495 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf6ecb-a7a8-41fa-ae8c-00cd1236896f None admin] PATH_INFO: `/v3/users/39d69e7aa10c4ac6a939422cb1abf944` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.557291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf6ecb-a7a8-41fa-ae8c-00cd1236896f None admin] RBAC: Authorizing `identity:get_user(user_id=39d69e7aa10c4ac6a939422cb1abf944)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:44.558781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf6ecb-a7a8-41fa-ae8c-00cd1236896f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:44.564045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 130/249] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:44 2022] GET /identity/v3/users/39d69e7aa10c4ac6a939422cb1abf944 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:44.564045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c150a-ed44-4e70-88be-566fdffaad8e None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:44.564045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c150a-ed44-4e70-88be-566fdffaad8e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:44.564045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c150a-ed44-4e70-88be-566fdffaad8e None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.564447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 131/250] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:44.564934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c150a-ed44-4e70-88be-566fdffaad8e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:44.624426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 120/251] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:44.625495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac6fbe-5cb9-4a71-a773-ca8918f899c7 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:44.625780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac6fbe-5cb9-4a71-a773-ca8918f899c7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:44.626052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac6fbe-5cb9-4a71-a773-ca8918f899c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.645344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f0ac6fbe-5cb9-4a71-a773-ca8918f899c7 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:44.646583 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f0ac6fbe-5cb9-4a71-a773-ca8918f899c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:44.659832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:44.695769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c150a-ed44-4e70-88be-566fdffaad8e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:44.709025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:44.733975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 121/252] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:44.735072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac6fbe-5cb9-4a71-a773-ca8918f899c7 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:44.743659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af28c6-d605-48bc-a5af-b6eef8fb0b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:44.745172 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af28c6-d605-48bc-a5af-b6eef8fb0b75 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:44.745661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af28c6-d605-48bc-a5af-b6eef8fb0b75 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:44.746134 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af28c6-d605-48bc-a5af-b6eef8fb0b75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.828062 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af28c6-d605-48bc-a5af-b6eef8fb0b75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:44.829573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af28c6-d605-48bc-a5af-b6eef8fb0b75 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:44.879862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac6fbe-5cb9-4a71-a773-ca8918f899c7 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:44.908527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:44.931607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cc80d-a0c4-4039-bc84-1bdcab413890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:44.933509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cc80d-a0c4-4039-bc84-1bdcab413890 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:44.934280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cc80d-a0c4-4039-bc84-1bdcab413890 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:44.934755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cc80d-a0c4-4039-bc84-1bdcab413890 None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:44.936300 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cc80d-a0c4-4039-bc84-1bdcab413890 None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:44.936951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 132/253] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:44.937935 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af28c6-d605-48bc-a5af-b6eef8fb0b75 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:44.938890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cc80d-a0c4-4039-bc84-1bdcab413890 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:44.942491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 122/254] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:44 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:44.943538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cc80d-a0c4-4039-bc84-1bdcab413890 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:45.014806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:45.043800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af28c6-d605-48bc-a5af-b6eef8fb0b75 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:45.062220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cc80d-a0c4-4039-bc84-1bdcab413890 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:45.079942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a100e5b-1f3f-41a4-b26f-23c4272bf648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:45.080784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a100e5b-1f3f-41a4-b26f-23c4272bf648 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:45.081065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a100e5b-1f3f-41a4-b26f-23c4272bf648 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:45.081337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a100e5b-1f3f-41a4-b26f-23c4272bf648 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:45.092071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a100e5b-1f3f-41a4-b26f-23c4272bf648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-5a100e5b-1f3f-41a4-b26f-23c4272bf648 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:45.100069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:45.100961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:45.101756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 133/255] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:35:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:45.102404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a100e5b-1f3f-41a4-b26f-23c4272bf648 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:45.108943 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ec0ac-d039-430c-b9f6-14f5a9e44403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:45.109779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ec0ac-d039-430c-b9f6-14f5a9e44403 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:45.110067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ec0ac-d039-430c-b9f6-14f5a9e44403 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:45.110350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ec0ac-d039-430c-b9f6-14f5a9e44403 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:45.123368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ec0ac-d039-430c-b9f6-14f5a9e44403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-f83ec0ac-d039-430c-b9f6-14f5a9e44403 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:45.133730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:45.134616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:45.135412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 123/256] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:35:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:45.136195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ec0ac-d039-430c-b9f6-14f5a9e44403 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:45.151653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:45.219793 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a100e5b-1f3f-41a4-b26f-23c4272bf648 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:45.246285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ec0ac-d039-430c-b9f6-14f5a9e44403 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:45.254464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b6fe0-3f54-40fc-b6ec-fdbaf394adaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:45.255563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b6fe0-3f54-40fc-b6ec-fdbaf394adaa None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:45.255675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b6fe0-3f54-40fc-b6ec-fdbaf394adaa None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:45.255885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b6fe0-3f54-40fc-b6ec-fdbaf394adaa None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:45.257117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b6fe0-3f54-40fc-b6ec-fdbaf394adaa None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:45.258594 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b6fe0-3f54-40fc-b6ec-fdbaf394adaa None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:45.260363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 134/257] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:45 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:45.261201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b6fe0-3f54-40fc-b6ec-fdbaf394adaa None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:45.264395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:45.286783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5534f65-0930-4776-8fb1-34b18de4c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:45.288187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5534f65-0930-4776-8fb1-34b18de4c28f None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:45.288671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5534f65-0930-4776-8fb1-34b18de4c28f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:45.289138 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5534f65-0930-4776-8fb1-34b18de4c28f None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62/users/39d69e7aa10c4ac6a939422cb1abf944/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:45.291516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5534f65-0930-4776-8fb1-34b18de4c28f None admin] RBAC: Authorizing `identity:create_grant(project_id=3bcbe8d7aaa3411b844ce5c42db14e62, user_id=39d69e7aa10c4ac6a939422cb1abf944, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:45.293361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5534f65-0930-4776-8fb1-34b18de4c28f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:45.308717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 124/258] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:35:45 2022] PUT /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62/users/39d69e7aa10c4ac6a939422cb1abf944/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:35:45.310134 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5534f65-0930-4776-8fb1-34b18de4c28f None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:45.369942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b6fe0-3f54-40fc-b6ec-fdbaf394adaa None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:45.371562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:45.407583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd734e8d-b886-44c8-b2d6-46c1dc089460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:45.408232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd734e8d-b886-44c8-b2d6-46c1dc089460 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:45.408577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd734e8d-b886-44c8-b2d6-46c1dc089460 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:45.408860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd734e8d-b886-44c8-b2d6-46c1dc089460 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:45.409682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd734e8d-b886-44c8-b2d6-46c1dc089460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:45.409996 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd734e8d-b886-44c8-b2d6-46c1dc089460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:45.411989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd734e8d-b886-44c8-b2d6-46c1dc089460 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:45.421120 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 135/259] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:45.421916 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd734e8d-b886-44c8-b2d6-46c1dc089460 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:45.446210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5534f65-0930-4776-8fb1-34b18de4c28f None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:45.484987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587f7ed-0602-464b-a088-2a9155e119b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:45.486250 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587f7ed-0602-464b-a088-2a9155e119b5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:45.486755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587f7ed-0602-464b-a088-2a9155e119b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:45.487235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587f7ed-0602-464b-a088-2a9155e119b5 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:45.488360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587f7ed-0602-464b-a088-2a9155e119b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:45.488951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587f7ed-0602-464b-a088-2a9155e119b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:45.490665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587f7ed-0602-464b-a088-2a9155e119b5 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:45.500580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 125/260] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:35:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:45.501776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587f7ed-0602-464b-a088-2a9155e119b5 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:45.541674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:45.554817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd734e8d-b886-44c8-b2d6-46c1dc089460 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:45.619898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587f7ed-0602-464b-a088-2a9155e119b5 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:45.629165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e163cb-433f-4a01-8c04-d6805d2bab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:45.629850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e163cb-433f-4a01-8c04-d6805d2bab64 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:45.630190 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e163cb-433f-4a01-8c04-d6805d2bab64 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:45.630484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e163cb-433f-4a01-8c04-d6805d2bab64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:45.631378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e163cb-433f-4a01-8c04-d6805d2bab64 None admin] RBAC: Adding query filter params (role.id=f740696dd1a847929b6236dcca34cc2f, scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62, user.id=9eebec6e44714f71bdd7a774873a6978) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:45.631674 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e163cb-433f-4a01-8c04-d6805d2bab64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:45.633184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e163cb-433f-4a01-8c04-d6805d2bab64 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:45.644335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 136/261] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:45 2022] GET /identity/v3/role_assignments?user.id=9eebec6e44714f71bdd7a774873a6978&scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62&role.id=f740696dd1a847929b6236dcca34cc2f => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:45.645237 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e163cb-433f-4a01-8c04-d6805d2bab64 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:45.666416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7442ae-c743-4a63-8831-02da0cd97ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:45.667370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7442ae-c743-4a63-8831-02da0cd97ec4 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:45.667656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7442ae-c743-4a63-8831-02da0cd97ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:45.667936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7442ae-c743-4a63-8831-02da0cd97ec4 None admin] PATH_INFO: `/v3/users/e72bf81dbd93487897393b43a6e4fdc9` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:45.669343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7442ae-c743-4a63-8831-02da0cd97ec4 None admin] RBAC: Authorizing `identity:get_user(user_id=e72bf81dbd93487897393b43a6e4fdc9)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:45.671009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7442ae-c743-4a63-8831-02da0cd97ec4 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:45.672794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 126/262] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:45 2022] GET /identity/v3/users/e72bf81dbd93487897393b43a6e4fdc9 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:45.678830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7442ae-c743-4a63-8831-02da0cd97ec4 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:45.691836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:45.843407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7442ae-c743-4a63-8831-02da0cd97ec4 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:45.844726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e163cb-433f-4a01-8c04-d6805d2bab64 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:45.856406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:45.905801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb485ef8-0b5e-432e-b9fa-4221c7199b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:45.907095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb485ef8-0b5e-432e-b9fa-4221c7199b2c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:45.907642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb485ef8-0b5e-432e-b9fa-4221c7199b2c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:45.908138 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb485ef8-0b5e-432e-b9fa-4221c7199b2c None admin] PATH_INFO: `/v3/users/e72bf81dbd93487897393b43a6e4fdc9` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:45.909715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb485ef8-0b5e-432e-b9fa-4221c7199b2c None admin] RBAC: Authorizing `identity:get_user(user_id=e72bf81dbd93487897393b43a6e4fdc9)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:45.911411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb485ef8-0b5e-432e-b9fa-4221c7199b2c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:45.913680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 137/263] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:45 2022] GET /identity/v3/users/e72bf81dbd93487897393b43a6e4fdc9 => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:45.920924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb485ef8-0b5e-432e-b9fa-4221c7199b2c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:45.927983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f305dc-aef8-447d-af3a-b97cfeb46c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:45.928978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f305dc-aef8-447d-af3a-b97cfeb46c90 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:45.929267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f305dc-aef8-447d-af3a-b97cfeb46c90 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:45.929552 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f305dc-aef8-447d-af3a-b97cfeb46c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:46.062712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f305dc-aef8-447d-af3a-b97cfeb46c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:46.064312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f305dc-aef8-447d-af3a-b97cfeb46c90 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:46.153307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:46.169086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb485ef8-0b5e-432e-b9fa-4221c7199b2c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:46.256811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ac636-986a-49cc-b177-4d2f185dbb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:46.257399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ac636-986a-49cc-b177-4d2f185dbb90 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:46.257788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ac636-986a-49cc-b177-4d2f185dbb90 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:46.258484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ac636-986a-49cc-b177-4d2f185dbb90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:46.298463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 127/264] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:46.304083 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f305dc-aef8-447d-af3a-b97cfeb46c90 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:46.312276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:46.416424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ac636-986a-49cc-b177-4d2f185dbb90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:46.418359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ac636-986a-49cc-b177-4d2f185dbb90 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:46.458755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f305dc-aef8-447d-af3a-b97cfeb46c90 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:46.518180 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da000bdd-0cff-4779-8271-5559e3d4ee2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:46.518990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da000bdd-0cff-4779-8271-5559e3d4ee2f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:46.519273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da000bdd-0cff-4779-8271-5559e3d4ee2f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:46.519547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da000bdd-0cff-4779-8271-5559e3d4ee2f None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:46.520776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da000bdd-0cff-4779-8271-5559e3d4ee2f None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:46.522391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da000bdd-0cff-4779-8271-5559e3d4ee2f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:46.524144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 128/265] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:46 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:46.530419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da000bdd-0cff-4779-8271-5559e3d4ee2f None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:46.630704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:46.650649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 138/266] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 730 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:46.658490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ac636-986a-49cc-b177-4d2f185dbb90 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:46.678721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:46.746140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da000bdd-0cff-4779-8271-5559e3d4ee2f None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:46.847286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ac636-986a-49cc-b177-4d2f185dbb90 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:46.848723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569349fc-9e48-4607-9576-a3b64de60be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:46.849406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569349fc-9e48-4607-9576-a3b64de60be0 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:46.849688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569349fc-9e48-4607-9576-a3b64de60be0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:46.849960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569349fc-9e48-4607-9576-a3b64de60be0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:46.850811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569349fc-9e48-4607-9576-a3b64de60be0 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62, user.id=e72bf81dbd93487897393b43a6e4fdc9) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:46.851123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569349fc-9e48-4607-9576-a3b64de60be0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:46.852625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569349fc-9e48-4607-9576-a3b64de60be0 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:46.866823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 129/267] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:46 2022] GET /identity/v3/role_assignments?user.id=e72bf81dbd93487897393b43a6e4fdc9&scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 608 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:46.868083 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a289de-7854-4389-9f82-f658c81924fb None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:46.868380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a289de-7854-4389-9f82-f658c81924fb None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:46.868680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a289de-7854-4389-9f82-f658c81924fb None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:46.870100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 130/268] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:46.879679 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33091a0-5af3-4324-b443-345fb35d9e2e None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:46.879967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33091a0-5af3-4324-b443-345fb35d9e2e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:46.880270 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33091a0-5af3-4324-b443-345fb35d9e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:46.904647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97600b-8b26-47cc-ba59-6a1d9997e6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:46.905556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97600b-8b26-47cc-ba59-6a1d9997e6f8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:46.905842 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97600b-8b26-47cc-ba59-6a1d9997e6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:46.906120 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97600b-8b26-47cc-ba59-6a1d9997e6f8 None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:46.907472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97600b-8b26-47cc-ba59-6a1d9997e6f8 None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:46.909026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97600b-8b26-47cc-ba59-6a1d9997e6f8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:46.910843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 139/269] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:46 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:46.912160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21d3bf-3cf8-4514-9407-552b5faeec96 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:46.912449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21d3bf-3cf8-4514-9407-552b5faeec96 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:46.912725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21d3bf-3cf8-4514-9407-552b5faeec96 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:46.914100 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 140/270] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:46.919189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21d3bf-3cf8-4514-9407-552b5faeec96 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:46.926323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a33091a0-5af3-4324-b443-345fb35d9e2e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:46.928609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a33091a0-5af3-4324-b443-345fb35d9e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:47.057126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:47.071776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 131/271] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:47.073343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361cadfd-3a3a-459b-9097-80e39a4744b3 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:47.073636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361cadfd-3a3a-459b-9097-80e39a4744b3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:47.073915 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361cadfd-3a3a-459b-9097-80e39a4744b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:47.097084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-361cadfd-3a3a-459b-9097-80e39a4744b3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:47.098952 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-361cadfd-3a3a-459b-9097-80e39a4744b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:47.109021 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21d3bf-3cf8-4514-9407-552b5faeec96 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:47.158702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8f30b-54c6-4e8b-8f16-90a57ff441d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:47.159426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8f30b-54c6-4e8b-8f16-90a57ff441d0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:47.159710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8f30b-54c6-4e8b-8f16-90a57ff441d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:47.160108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8f30b-54c6-4e8b-8f16-90a57ff441d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:47.160957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8f30b-54c6-4e8b-8f16-90a57ff441d0 None admin] RBAC: Adding query filter params (role.id=ff6066b1fb8940cd9ec82c078fd8d2f6, scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62, user.id=e72bf81dbd93487897393b43a6e4fdc9) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:47.161875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8f30b-54c6-4e8b-8f16-90a57ff441d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:47.163428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8f30b-54c6-4e8b-8f16-90a57ff441d0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:47.170354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 141/272] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:46 2022] GET /identity/v3/role_assignments?user.id=e72bf81dbd93487897393b43a6e4fdc9&scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62&role.id=ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:47.171205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8f30b-54c6-4e8b-8f16-90a57ff441d0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:47.196447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:47.208683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 132/273] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:47.251346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361cadfd-3a3a-459b-9097-80e39a4744b3 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:47.333975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8f30b-54c6-4e8b-8f16-90a57ff441d0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:47.382383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:47.399463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e9d0c-7b60-45d7-8c87-299da648a1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:47.400299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e9d0c-7b60-45d7-8c87-299da648a1e5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:47.400576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e9d0c-7b60-45d7-8c87-299da648a1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:47.400844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e9d0c-7b60-45d7-8c87-299da648a1e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:47.402340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e9d0c-7b60-45d7-8c87-299da648a1e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:47.403799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e9d0c-7b60-45d7-8c87-299da648a1e5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:47.407589 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 142/274] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:47.415024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e9d0c-7b60-45d7-8c87-299da648a1e5 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:47.423270 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361cadfd-3a3a-459b-9097-80e39a4744b3 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:47.471789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657b8d6-47fb-4f44-8ad6-cccb617bd44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:47.472917 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657b8d6-47fb-4f44-8ad6-cccb617bd44f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:47.473343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657b8d6-47fb-4f44-8ad6-cccb617bd44f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:47.473862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657b8d6-47fb-4f44-8ad6-cccb617bd44f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:47.487464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657b8d6-47fb-4f44-8ad6-cccb617bd44f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-c657b8d6-47fb-4f44-8ad6-cccb617bd44f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:47.496268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:47.497265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:47.498087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:47.498087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:47.498087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:47.498087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:47.498087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:47.498087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:47.498087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:47.498087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:47.498087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:47.498087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:47.498087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:47.498087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:47.500838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 133/275] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:47.508042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657b8d6-47fb-4f44-8ad6-cccb617bd44f None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:47.545928 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e9d0c-7b60-45d7-8c87-299da648a1e5 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:47.568919 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:47.592783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1c312-5c86-494b-b5d8-5b7e2eba86a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:47.593467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1c312-5c86-494b-b5d8-5b7e2eba86a0 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:47.594291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1c312-5c86-494b-b5d8-5b7e2eba86a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:47.594578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1c312-5c86-494b-b5d8-5b7e2eba86a0 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:47.595329 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1c312-5c86-494b-b5d8-5b7e2eba86a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:47.597493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1c312-5c86-494b-b5d8-5b7e2eba86a0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:47.622811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657b8d6-47fb-4f44-8ad6-cccb617bd44f None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:47.625774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 143/276] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:35:47 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:47.664426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c4cb8-6c49-4bdc-b42a-ca10c4636a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:47.665089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c4cb8-6c49-4bdc-b42a-ca10c4636a7f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:47.665381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c4cb8-6c49-4bdc-b42a-ca10c4636a7f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:47.665657 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c4cb8-6c49-4bdc-b42a-ca10c4636a7f None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:47.666433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c4cb8-6c49-4bdc-b42a-ca10c4636a7f None admin] RBAC: Adding query filter params (name=member) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:47.666812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c4cb8-6c49-4bdc-b42a-ca10c4636a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:47.668458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c4cb8-6c49-4bdc-b42a-ca10c4636a7f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:47.679004 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 134/277] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:47.685248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1c312-5c86-494b-b5d8-5b7e2eba86a0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:47.746958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:47.801460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1c312-5c86-494b-b5d8-5b7e2eba86a0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:47.835907 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa13e17-08f0-4f8e-a74b-059d439736b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:47.836882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa13e17-08f0-4f8e-a74b-059d439736b4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:47.837169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa13e17-08f0-4f8e-a74b-059d439736b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:47.837446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa13e17-08f0-4f8e-a74b-059d439736b4 None admin] PATH_INFO: `/v3/users/836c5b1a1a33481693f7da6349a8dd29` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:47.851954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa13e17-08f0-4f8e-a74b-059d439736b4 None admin] RBAC: Authorizing `identity:get_user(user_id=836c5b1a1a33481693f7da6349a8dd29)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:47.853470 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa13e17-08f0-4f8e-a74b-059d439736b4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:47.855339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 144/278] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:47 2022] GET /identity/v3/users/836c5b1a1a33481693f7da6349a8dd29 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:47.860936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c4cb8-6c49-4bdc-b42a-ca10c4636a7f None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:47.971227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c4cb8-6c49-4bdc-b42a-ca10c4636a7f None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:47.990427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:48.010744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45ab87-d503-4299-a2b1-0dd4de2c3176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:48.011708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45ab87-d503-4299-a2b1-0dd4de2c3176 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:48.011992 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45ab87-d503-4299-a2b1-0dd4de2c3176 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:48.012363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45ab87-d503-4299-a2b1-0dd4de2c3176 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:48.013197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45ab87-d503-4299-a2b1-0dd4de2c3176 None admin] RBAC: Adding query filter params (role.id=8f3ea95f038d4d8fbd1fd4897be36344, user.id=836c5b1a1a33481693f7da6349a8dd29, scope.system=all) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:48.013513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45ab87-d503-4299-a2b1-0dd4de2c3176 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:48.014999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45ab87-d503-4299-a2b1-0dd4de2c3176 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:48.023472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 135/279] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Jul 12 03:35:47 2022] GET /identity/v3/role_assignments?user.id=836c5b1a1a33481693f7da6349a8dd29&scope.system=all&role.id=8f3ea95f038d4d8fbd1fd4897be36344 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:48.074815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d26e0-cb56-4acd-9d0f-a801ee17ca09 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:48.075144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d26e0-cb56-4acd-9d0f-a801ee17ca09 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:48.075407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489d26e0-cb56-4acd-9d0f-a801ee17ca09 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:48.077081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 145/280] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:48.082506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4876c-3b97-4ac1-9719-e906b1dbd2ac None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:48.082877 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4876c-3b97-4ac1-9719-e906b1dbd2ac None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:48.083064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4876c-3b97-4ac1-9719-e906b1dbd2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:48.100972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a0b4876c-3b97-4ac1-9719-e906b1dbd2ac None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:48.102183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a0b4876c-3b97-4ac1-9719-e906b1dbd2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:48.176153 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 136/281] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:48.198963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d26e0-cb56-4acd-9d0f-a801ee17ca09 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:48.320021 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489d26e0-cb56-4acd-9d0f-a801ee17ca09 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:48.340336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:48.367715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb935970-caad-48c8-824f-8f6b26991d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:48.368560 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb935970-caad-48c8-824f-8f6b26991d20 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:48.368844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb935970-caad-48c8-824f-8f6b26991d20 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:48.369116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb935970-caad-48c8-824f-8f6b26991d20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:48.383555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb935970-caad-48c8-824f-8f6b26991d20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-bb935970-caad-48c8-824f-8f6b26991d20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:48.393751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:48.394616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:48.395827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 146/282] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:48.405151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4876c-3b97-4ac1-9719-e906b1dbd2ac None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:48.494177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7816dd-8482-41ea-86b6-247519ac676c None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:48.494525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7816dd-8482-41ea-86b6-247519ac676c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:48.494733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7816dd-8482-41ea-86b6-247519ac676c None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:48.496363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 147/283] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:48.501301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8a512-38f3-483c-bfd9-b7f5896fe8e3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:48.501584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8a512-38f3-483c-bfd9-b7f5896fe8e3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:48.501856 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8a512-38f3-483c-bfd9-b7f5896fe8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:48.519115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2cc8a512-38f3-483c-bfd9-b7f5896fe8e3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:48.520367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2cc8a512-38f3-483c-bfd9-b7f5896fe8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:48.595490 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4876c-3b97-4ac1-9719-e906b1dbd2ac None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:48.620380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:48.629783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 148/284] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:48.661041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115cb74-8cfe-4f05-be84-61e173135a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:48.661717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115cb74-8cfe-4f05-be84-61e173135a66 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:48.661999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115cb74-8cfe-4f05-be84-61e173135a66 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:48.662381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115cb74-8cfe-4f05-be84-61e173135a66 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:48.663676 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115cb74-8cfe-4f05-be84-61e173135a66 None admin] RBAC: Adding query filter params (name=member) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:48.664238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115cb74-8cfe-4f05-be84-61e173135a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:48.666518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115cb74-8cfe-4f05-be84-61e173135a66 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:48.679649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 137/285] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:48.682636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8a512-38f3-483c-bfd9-b7f5896fe8e3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:48.693047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115cb74-8cfe-4f05-be84-61e173135a66 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:48.841606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8a512-38f3-483c-bfd9-b7f5896fe8e3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:48.844301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:48.876024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb2c4a-e3bf-48a9-b85a-3049a51e9f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:48.876770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb2c4a-e3bf-48a9-b85a-3049a51e9f14 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:48.877364 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb2c4a-e3bf-48a9-b85a-3049a51e9f14 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:48.877734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb2c4a-e3bf-48a9-b85a-3049a51e9f14 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:48.882929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115cb74-8cfe-4f05-be84-61e173135a66 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:48.888311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb2c4a-e3bf-48a9-b85a-3049a51e9f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-26bb2c4a-e3bf-48a9-b85a-3049a51e9f14 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:48.902832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:48.903728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:48.904523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 149/286] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:48.913035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb2c4a-e3bf-48a9-b85a-3049a51e9f14 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:48.961585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d8cb2-4a9b-4d8e-885f-c92033c8589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:48.961939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d8cb2-4a9b-4d8e-885f-c92033c8589a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:48.962223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d8cb2-4a9b-4d8e-885f-c92033c8589a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:48.962487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d8cb2-4a9b-4d8e-885f-c92033c8589a None admin] PATH_INFO: `/v3/users/39d69e7aa10c4ac6a939422cb1abf944` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:48.963869 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d8cb2-4a9b-4d8e-885f-c92033c8589a None admin] RBAC: Authorizing `identity:get_user(user_id=39d69e7aa10c4ac6a939422cb1abf944)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:48.965374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d8cb2-4a9b-4d8e-885f-c92033c8589a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:48.967198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 138/287] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:48 2022] GET /identity/v3/users/39d69e7aa10c4ac6a939422cb1abf944 => generated 297 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:48.972945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d8cb2-4a9b-4d8e-885f-c92033c8589a None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:49.059535 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:49.107768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb2c4a-e3bf-48a9-b85a-3049a51e9f14 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:49.153878 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d8cb2-4a9b-4d8e-885f-c92033c8589a None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:49.167915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5dbc7a-e4a8-4d02-97d2-d7a0b50c2edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:49.168642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dbc7a-e4a8-4d02-97d2-d7a0b50c2edd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:49.168922 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dbc7a-e4a8-4d02-97d2-d7a0b50c2edd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:49.169201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5dbc7a-e4a8-4d02-97d2-d7a0b50c2edd None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:49.170016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5dbc7a-e4a8-4d02-97d2-d7a0b50c2edd None admin] RBAC: Adding query filter params (name=reader) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:49.170325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5dbc7a-e4a8-4d02-97d2-d7a0b50c2edd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:49.171924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5dbc7a-e4a8-4d02-97d2-d7a0b50c2edd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:49.195091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 150/288] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:49.205132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5dbc7a-e4a8-4d02-97d2-d7a0b50c2edd None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:49.232376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:49.263440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c10532-9697-45fc-8399-dfff8482a20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:49.264130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10532-9697-45fc-8399-dfff8482a20b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:49.264414 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10532-9697-45fc-8399-dfff8482a20b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:49.264683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10532-9697-45fc-8399-dfff8482a20b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:49.311008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5dbc7a-e4a8-4d02-97d2-d7a0b50c2edd None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:49.376398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7311c-7545-4158-8da2-df6646900160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:49.377282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7311c-7545-4158-8da2-df6646900160 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:49.377566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7311c-7545-4158-8da2-df6646900160 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:49.377839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7311c-7545-4158-8da2-df6646900160 None admin] PATH_INFO: `/v3/users/9eebec6e44714f71bdd7a774873a6978` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:49.381024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7311c-7545-4158-8da2-df6646900160 None admin] RBAC: Authorizing `identity:get_user(user_id=9eebec6e44714f71bdd7a774873a6978)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:49.384604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7311c-7545-4158-8da2-df6646900160 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:49.390679 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 151/289] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:49 2022] GET /identity/v3/users/9eebec6e44714f71bdd7a774873a6978 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:49.394311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c10532-9697-45fc-8399-dfff8482a20b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:49.396452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7311c-7545-4158-8da2-df6646900160 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:49.405071 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c10532-9697-45fc-8399-dfff8482a20b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:49.472420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:49.519841 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 139/290] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:49.525229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c10532-9697-45fc-8399-dfff8482a20b None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:49.604357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:49.617444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7311c-7545-4158-8da2-df6646900160 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:49.629429 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c10532-9697-45fc-8399-dfff8482a20b None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:49.664603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a54ce-f5ee-458d-ba00-241557383e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:49.665506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a54ce-f5ee-458d-ba00-241557383e70 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:49.665787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a54ce-f5ee-458d-ba00-241557383e70 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:49.666098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a54ce-f5ee-458d-ba00-241557383e70 None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:49.667278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a54ce-f5ee-458d-ba00-241557383e70 None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:49.668770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a54ce-f5ee-458d-ba00-241557383e70 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:49.670770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 140/291] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:49 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:49.676582 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a54ce-f5ee-458d-ba00-241557383e70 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:49.689543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:49.693223 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638a838-a46b-4f2b-be6b-b50803a8c0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:49.694217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638a838-a46b-4f2b-be6b-b50803a8c0b6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:49.694504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638a838-a46b-4f2b-be6b-b50803a8c0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:49.694782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638a838-a46b-4f2b-be6b-b50803a8c0b6 None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:49.700849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638a838-a46b-4f2b-be6b-b50803a8c0b6 None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:49.702444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638a838-a46b-4f2b-be6b-b50803a8c0b6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:49.704750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 152/292] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:49 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:49.710554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638a838-a46b-4f2b-be6b-b50803a8c0b6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:49.783432 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a54ce-f5ee-458d-ba00-241557383e70 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:49.820805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c4362-5924-4b3d-a59a-869f2475dafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:49.821733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4362-5924-4b3d-a59a-869f2475dafb None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:49.822017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4362-5924-4b3d-a59a-869f2475dafb None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:49.822293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4362-5924-4b3d-a59a-869f2475dafb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:49.823279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c4362-5924-4b3d-a59a-869f2475dafb None admin] RBAC: Adding query filter params (role.id=8f3ea95f038d4d8fbd1fd4897be36344, scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62, user.id=39d69e7aa10c4ac6a939422cb1abf944) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:49.823593 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c4362-5924-4b3d-a59a-869f2475dafb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:49.825084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c4362-5924-4b3d-a59a-869f2475dafb None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:49.831977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 141/293] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:49 2022] GET /identity/v3/role_assignments?user.id=39d69e7aa10c4ac6a939422cb1abf944&scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62&role.id=8f3ea95f038d4d8fbd1fd4897be36344 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:49.936490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638a838-a46b-4f2b-be6b-b50803a8c0b6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:49.990834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04dd9a-b2fb-4f36-90b1-9830a4d2a64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:49.991745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04dd9a-b2fb-4f36-90b1-9830a4d2a64c None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:49.992045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04dd9a-b2fb-4f36-90b1-9830a4d2a64c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:49.992327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04dd9a-b2fb-4f36-90b1-9830a4d2a64c None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62/users/9eebec6e44714f71bdd7a774873a6978/roles/f740696dd1a847929b6236dcca34cc2f` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:49.994408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04dd9a-b2fb-4f36-90b1-9830a4d2a64c None admin] RBAC: Authorizing `identity:create_grant(project_id=3bcbe8d7aaa3411b844ce5c42db14e62, user_id=9eebec6e44714f71bdd7a774873a6978, role_id=f740696dd1a847929b6236dcca34cc2f)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:49.995918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c04dd9a-b2fb-4f36-90b1-9830a4d2a64c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:50.016829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 153/294] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:35:49 2022] PUT /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62/users/9eebec6e44714f71bdd7a774873a6978/roles/f740696dd1a847929b6236dcca34cc2f => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:35:50.169550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfaaf6-41f0-44bc-89a6-58da48e6a5a5 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:50.169904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfaaf6-41f0-44bc-89a6-58da48e6a5a5 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:50.170110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfaaf6-41f0-44bc-89a6-58da48e6a5a5 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:50.171733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 142/295] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:50.177563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24c74a-60bc-4f6b-8abc-bdd534ef2b13 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:50.178139 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24c74a-60bc-4f6b-8abc-bdd534ef2b13 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:50.178668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24c74a-60bc-4f6b-8abc-bdd534ef2b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:50.197806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5d24c74a-60bc-4f6b-8abc-bdd534ef2b13 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:50.199511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5d24c74a-60bc-4f6b-8abc-bdd534ef2b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:50.215927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:50.278056 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 154/296] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:50.301939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfaaf6-41f0-44bc-89a6-58da48e6a5a5 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:50.380041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:50.405969 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfaaf6-41f0-44bc-89a6-58da48e6a5a5 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:50.408028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a873f-6084-4e86-a5a2-d29427a3e21f None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:50.408312 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a873f-6084-4e86-a5a2-d29427a3e21f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:50.408581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a873f-6084-4e86-a5a2-d29427a3e21f None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:50.410070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 155/297] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:50.416582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef89ae-3cb0-4c50-a616-cacf3e5f9f9a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:50.416825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef89ae-3cb0-4c50-a616-cacf3e5f9f9a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:50.417300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef89ae-3cb0-4c50-a616-cacf3e5f9f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:50.435216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dcef89ae-3cb0-4c50-a616-cacf3e5f9f9a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:50.436574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dcef89ae-3cb0-4c50-a616-cacf3e5f9f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:50.450735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967fe58-9a7d-403f-8756-0d4a9a598f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:50.452027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967fe58-9a7d-403f-8756-0d4a9a598f25 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:50.452611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967fe58-9a7d-403f-8756-0d4a9a598f25 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:50.453049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967fe58-9a7d-403f-8756-0d4a9a598f25 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:50.464425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967fe58-9a7d-403f-8756-0d4a9a598f25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-5967fe58-9a7d-403f-8756-0d4a9a598f25 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:50.473519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:50.474433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:50.475212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:50.475212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:50.475212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:50.475212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:50.475212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:50.475212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:50.475212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:50.475212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:50.475212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:50.475212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:50.475212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:50.475212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:50.477991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 143/298] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:50.484166 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967fe58-9a7d-403f-8756-0d4a9a598f25 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:50.508419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 156/299] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:50.530506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef89ae-3cb0-4c50-a616-cacf3e5f9f9a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:50.593216 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967fe58-9a7d-403f-8756-0d4a9a598f25 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:50.608932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:50.629442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890375fd-1b38-4803-8bc8-3888a254443c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:50.630697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890375fd-1b38-4803-8bc8-3888a254443c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:50.630997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890375fd-1b38-4803-8bc8-3888a254443c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:50.631406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890375fd-1b38-4803-8bc8-3888a254443c None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:50.632316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890375fd-1b38-4803-8bc8-3888a254443c None admin] RBAC: Adding query filter params (name=reader) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:50.632627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890375fd-1b38-4803-8bc8-3888a254443c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:50.633487 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef89ae-3cb0-4c50-a616-cacf3e5f9f9a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:50.634072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890375fd-1b38-4803-8bc8-3888a254443c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:50.635747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:50.649443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 144/300] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:50.649443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7657f-2c64-4d2f-ac7e-d63c1fb19c09 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:50.649443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7657f-2c64-4d2f-ac7e-d63c1fb19c09 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:50.649443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7657f-2c64-4d2f-ac7e-d63c1fb19c09 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:50.649973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 145/301] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:50.650903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7657f-2c64-4d2f-ac7e-d63c1fb19c09 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:50.673815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432495d1-b168-4ad5-b9ad-d3e2ba1929ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:50.674611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432495d1-b168-4ad5-b9ad-d3e2ba1929ec None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:50.674887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432495d1-b168-4ad5-b9ad-d3e2ba1929ec None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:50.675157 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432495d1-b168-4ad5-b9ad-d3e2ba1929ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:50.676351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432495d1-b168-4ad5-b9ad-d3e2ba1929ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:50.677797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432495d1-b168-4ad5-b9ad-d3e2ba1929ec None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:50.679406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 157/302] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:50.680541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49ab00-567e-49d3-bf73-e3c058a9f1a8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:50.680823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49ab00-567e-49d3-bf73-e3c058a9f1a8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:50.681091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49ab00-567e-49d3-bf73-e3c058a9f1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:50.697467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1a49ab00-567e-49d3-bf73-e3c058a9f1a8 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:50.698781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1a49ab00-567e-49d3-bf73-e3c058a9f1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:50.765432 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7657f-2c64-4d2f-ac7e-d63c1fb19c09 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:50.780068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 158/303] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:50.780068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49ab00-567e-49d3-bf73-e3c058a9f1a8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:50.809857 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adc2da-3cfd-40a6-ac59-051852f529dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:50.810744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adc2da-3cfd-40a6-ac59-051852f529dd None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:50.811028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adc2da-3cfd-40a6-ac59-051852f529dd None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:50.811418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adc2da-3cfd-40a6-ac59-051852f529dd None admin] PATH_INFO: `/v3/users/f28344bd28f549f2bbc87b5d6b2ab5fd` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:50.826880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adc2da-3cfd-40a6-ac59-051852f529dd None admin] RBAC: Authorizing `identity:get_user(user_id=f28344bd28f549f2bbc87b5d6b2ab5fd)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:50.828619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adc2da-3cfd-40a6-ac59-051852f529dd None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:50.830682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 146/304] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:50 2022] GET /identity/v3/users/f28344bd28f549f2bbc87b5d6b2ab5fd => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:50.831341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adc2da-3cfd-40a6-ac59-051852f529dd None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:50.861755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:50.889552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49ab00-567e-49d3-bf73-e3c058a9f1a8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:50.891650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:50.926372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7e55d-bd40-4587-bf1f-5a534f3ae2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:50.927038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7e55d-bd40-4587-bf1f-5a534f3ae2a3 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:50.927322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7e55d-bd40-4587-bf1f-5a534f3ae2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:50.927595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7e55d-bd40-4587-bf1f-5a534f3ae2a3 None admin] PATH_INFO: `/v3/groups` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:50.928309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7e55d-bd40-4587-bf1f-5a534f3ae2a3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:50.929774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7e55d-bd40-4587-bf1f-5a534f3ae2a3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:50.944785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adc2da-3cfd-40a6-ac59-051852f529dd None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:50.959169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 159/305] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Jul 12 03:35:50 2022] POST /identity/v3/groups => generated 229 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:50.959169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7e55d-bd40-4587-bf1f-5a534f3ae2a3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:50.997029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1be852-c5f0-4eb6-a974-25cc722fc1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:50.997883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1be852-c5f0-4eb6-a974-25cc722fc1e1 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:50.998168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1be852-c5f0-4eb6-a974-25cc722fc1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:50.998488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1be852-c5f0-4eb6-a974-25cc722fc1e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:51.010703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1be852-c5f0-4eb6-a974-25cc722fc1e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-eb1be852-c5f0-4eb6-a974-25cc722fc1e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:51.019663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:51.020741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:51.022155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 147/306] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:51.027482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1be852-c5f0-4eb6-a974-25cc722fc1e1 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:51.059355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7e55d-bd40-4587-bf1f-5a534f3ae2a3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:51.106812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9435-c676-4542-9b63-410dbc544ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:51.107523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9435-c676-4542-9b63-410dbc544ca3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:51.108039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9435-c676-4542-9b63-410dbc544ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:51.108470 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9435-c676-4542-9b63-410dbc544ca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:51.109342 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad9435-c676-4542-9b63-410dbc544ca3 None admin] RBAC: Adding query filter params (role.id=f740696dd1a847929b6236dcca34cc2f, user.id=f28344bd28f549f2bbc87b5d6b2ab5fd, scope.system=all) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:51.109656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad9435-c676-4542-9b63-410dbc544ca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:51.112080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad9435-c676-4542-9b63-410dbc544ca3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:51.115545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:51.121893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 160/307] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Jul 12 03:35:50 2022] GET /identity/v3/role_assignments?user.id=f28344bd28f549f2bbc87b5d6b2ab5fd&scope.system=all&role.id=f740696dd1a847929b6236dcca34cc2f => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:51.168933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1be852-c5f0-4eb6-a974-25cc722fc1e1 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:51.187911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:51.211405 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4b90f-ba69-4fe9-9b58-8ba85673077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:51.212121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4b90f-ba69-4fe9-9b58-8ba85673077f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:51.212404 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4b90f-ba69-4fe9-9b58-8ba85673077f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:51.212678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4b90f-ba69-4fe9-9b58-8ba85673077f None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:51.213457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4b90f-ba69-4fe9-9b58-8ba85673077f None admin] RBAC: Adding query filter params (name=member) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:51.213764 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4b90f-ba69-4fe9-9b58-8ba85673077f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:51.215328 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4b90f-ba69-4fe9-9b58-8ba85673077f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:51.225974 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 148/308] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:51.231788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9435-c676-4542-9b63-410dbc544ca3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:51.346424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9435-c676-4542-9b63-410dbc544ca3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:51.381117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e83279-1e0a-4b3e-989a-3e2a0aed21c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:51.382236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e83279-1e0a-4b3e-989a-3e2a0aed21c7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:51.382674 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e83279-1e0a-4b3e-989a-3e2a0aed21c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:51.383088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e83279-1e0a-4b3e-989a-3e2a0aed21c7 None admin] PATH_INFO: `/v3/users/836c5b1a1a33481693f7da6349a8dd29` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:51.384452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e83279-1e0a-4b3e-989a-3e2a0aed21c7 None admin] RBAC: Authorizing `identity:get_user(user_id=836c5b1a1a33481693f7da6349a8dd29)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:51.386042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e83279-1e0a-4b3e-989a-3e2a0aed21c7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:51.387801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 161/309] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:51 2022] GET /identity/v3/users/836c5b1a1a33481693f7da6349a8dd29 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:51.393912 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4b90f-ba69-4fe9-9b58-8ba85673077f None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:51.493444 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4b90f-ba69-4fe9-9b58-8ba85673077f None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:51.531377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeab856-eb6e-4446-9dd0-64dc68b1e93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:51.532518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeab856-eb6e-4446-9dd0-64dc68b1e93e None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:51.532935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeab856-eb6e-4446-9dd0-64dc68b1e93e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:51.533367 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeab856-eb6e-4446-9dd0-64dc68b1e93e None admin] PATH_INFO: `/v3/system/users/836c5b1a1a33481693f7da6349a8dd29/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:51.534969 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeab856-eb6e-4446-9dd0-64dc68b1e93e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=836c5b1a1a33481693f7da6349a8dd29, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:51.536517 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeab856-eb6e-4446-9dd0-64dc68b1e93e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:51.547269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 149/310] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Jul 12 03:35:51 2022] PUT /identity/v3/system/users/836c5b1a1a33481693f7da6349a8dd29/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:35:52.133652 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded3071-912b-426d-93fe-591e71beb341 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:52.134166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded3071-912b-426d-93fe-591e71beb341 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:52.134268 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded3071-912b-426d-93fe-591e71beb341 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:52.135717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 162/311] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:52.141374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cb20b-a88a-4f9b-821b-742d469ef7b6 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:52.141778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cb20b-a88a-4f9b-821b-742d469ef7b6 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:52.141979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cb20b-a88a-4f9b-821b-742d469ef7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:52.162017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-552cb20b-a88a-4f9b-821b-742d469ef7b6 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:52.163347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-552cb20b-a88a-4f9b-821b-742d469ef7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:52.201621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:52.235620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 150/312] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:52.257904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded3071-912b-426d-93fe-591e71beb341 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:52.332169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:52.371460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded3071-912b-426d-93fe-591e71beb341 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:52.406315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764b365-10ae-4936-844d-521ee1912e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:52.407387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764b365-10ae-4936-844d-521ee1912e60 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:52.407883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764b365-10ae-4936-844d-521ee1912e60 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:52.408383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764b365-10ae-4936-844d-521ee1912e60 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:52.418953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764b365-10ae-4936-844d-521ee1912e60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e764b365-10ae-4936-844d-521ee1912e60 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:52.426865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:52.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:52.428545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:52.428545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:52.428545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:52.428545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:52.428545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:52.428545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:52.428545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:52.428545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:52.428545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:52.428545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:52.428545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:52.428545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:52.430899 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 163/313] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:52.437119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cb20b-a88a-4f9b-821b-742d469ef7b6 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:52.537350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cb20b-a88a-4f9b-821b-742d469ef7b6 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:52.570175 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ba61d-ad17-4ad0-8430-a9e95f2b6484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:52.570845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ba61d-ad17-4ad0-8430-a9e95f2b6484 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:52.571127 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ba61d-ad17-4ad0-8430-a9e95f2b6484 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:52.571400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ba61d-ad17-4ad0-8430-a9e95f2b6484 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:52.572173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ba61d-ad17-4ad0-8430-a9e95f2b6484 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:52.572478 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ba61d-ad17-4ad0-8430-a9e95f2b6484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:52.573960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ba61d-ad17-4ad0-8430-a9e95f2b6484 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:52.586116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 151/314] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:52.591831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764b365-10ae-4936-844d-521ee1912e60 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:52.695785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764b365-10ae-4936-844d-521ee1912e60 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:52.724119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:52.732649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f8e66-3468-41a7-b6ba-a23702b50e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:52.733539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f8e66-3468-41a7-b6ba-a23702b50e55 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:52.733823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f8e66-3468-41a7-b6ba-a23702b50e55 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:52.734097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f8e66-3468-41a7-b6ba-a23702b50e55 None admin] PATH_INFO: `/v3/users/9eebec6e44714f71bdd7a774873a6978` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:52.735321 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f8e66-3468-41a7-b6ba-a23702b50e55 None admin] RBAC: Authorizing `identity:get_user(user_id=9eebec6e44714f71bdd7a774873a6978)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:52.736833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f8e66-3468-41a7-b6ba-a23702b50e55 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:52.739087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 164/315] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:52 2022] GET /identity/v3/users/9eebec6e44714f71bdd7a774873a6978 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:52.745053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ba61d-ad17-4ad0-8430-a9e95f2b6484 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:52.781163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:52.850437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ba61d-ad17-4ad0-8430-a9e95f2b6484 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:52.888798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f3ac4-8352-4201-a5d2-0e354565dbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:52.889454 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f3ac4-8352-4201-a5d2-0e354565dbfc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:52.889735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f3ac4-8352-4201-a5d2-0e354565dbfc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:52.890008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f3ac4-8352-4201-a5d2-0e354565dbfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:52.958532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f3ac4-8352-4201-a5d2-0e354565dbfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:52.960287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f3ac4-8352-4201-a5d2-0e354565dbfc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:52.990961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:53.063153 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 152/316] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:53.064568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f8e66-3468-41a7-b6ba-a23702b50e55 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:53.117458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829cf45-7cbf-43e7-8607-bf27c1fc6e2d None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:53.117740 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829cf45-7cbf-43e7-8607-bf27c1fc6e2d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:53.117933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829cf45-7cbf-43e7-8607-bf27c1fc6e2d None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:53.119659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 153/317] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:35:53.124509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa987249-1f6a-4686-9fb5-e30018dafea5 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:53.124792 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa987249-1f6a-4686-9fb5-e30018dafea5 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:53.125060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa987249-1f6a-4686-9fb5-e30018dafea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:53.141617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-fa987249-1f6a-4686-9fb5-e30018dafea5 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:53.142782 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-fa987249-1f6a-4686-9fb5-e30018dafea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:53.169002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f8e66-3468-41a7-b6ba-a23702b50e55 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:53.213471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 154/318] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:53.221251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fde3f-8bec-4e9a-88bd-5420655745ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:53.222137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fde3f-8bec-4e9a-88bd-5420655745ca None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:53.222420 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fde3f-8bec-4e9a-88bd-5420655745ca None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:53.222695 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fde3f-8bec-4e9a-88bd-5420655745ca None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:53.223885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fde3f-8bec-4e9a-88bd-5420655745ca None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:53.225436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fde3f-8bec-4e9a-88bd-5420655745ca None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:53.227408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 165/319] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:53 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:53.233071 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa987249-1f6a-4686-9fb5-e30018dafea5 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:53.236834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fde3f-8bec-4e9a-88bd-5420655745ca None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:53.297235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:53.308391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:53.336570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa987249-1f6a-4686-9fb5-e30018dafea5 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:53.350028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fde3f-8bec-4e9a-88bd-5420655745ca None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:53.372350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b19de-9e49-4cd7-99bc-b8a6f8ecb864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:53.373260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b19de-9e49-4cd7-99bc-b8a6f8ecb864 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:53.373537 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b19de-9e49-4cd7-99bc-b8a6f8ecb864 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:53.373821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b19de-9e49-4cd7-99bc-b8a6f8ecb864 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:53.374900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b19de-9e49-4cd7-99bc-b8a6f8ecb864 None admin] RBAC: Adding query filter params (role.id=f740696dd1a847929b6236dcca34cc2f, scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62, user.id=9eebec6e44714f71bdd7a774873a6978) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:53.375029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b19de-9e49-4cd7-99bc-b8a6f8ecb864 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:53.376558 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b19de-9e49-4cd7-99bc-b8a6f8ecb864 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:53.390285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 155/320] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:35:53 2022] GET /identity/v3/role_assignments?user.id=9eebec6e44714f71bdd7a774873a6978&scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62&role.id=f740696dd1a847929b6236dcca34cc2f => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:53.390285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f4f87-73d5-4b0a-9398-64e23d24566d None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:53.390285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f4f87-73d5-4b0a-9398-64e23d24566d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:53.390285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f4f87-73d5-4b0a-9398-64e23d24566d None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:53.390285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 156/321] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:53.392118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb6aac-9144-4b38-be91-f27e90a96a25 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:53.392412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb6aac-9144-4b38-be91-f27e90a96a25 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:53.392685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb6aac-9144-4b38-be91-f27e90a96a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:53.394045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f440e7-642a-467c-b633-6af50fe28a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:53.394931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f440e7-642a-467c-b633-6af50fe28a56 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:53.395305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f440e7-642a-467c-b633-6af50fe28a56 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:53.395666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f440e7-642a-467c-b633-6af50fe28a56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:53.397010 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f440e7-642a-467c-b633-6af50fe28a56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:53.398625 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f440e7-642a-467c-b633-6af50fe28a56 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:53.401429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 166/322] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:35:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:53.409319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f440e7-642a-467c-b633-6af50fe28a56 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:53.415725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f4bb6aac-9144-4b38-be91-f27e90a96a25 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:53.417058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f4bb6aac-9144-4b38-be91-f27e90a96a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:53.510298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 157/323] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:53.527931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f440e7-642a-467c-b633-6af50fe28a56 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:53.533744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb6aac-9144-4b38-be91-f27e90a96a25 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:53.565817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:53.573277 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a36038-4d1f-42e0-85cf-356919c64849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:53.573955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a36038-4d1f-42e0-85cf-356919c64849 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:53.574243 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a36038-4d1f-42e0-85cf-356919c64849 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:53.574517 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a36038-4d1f-42e0-85cf-356919c64849 None admin] PATH_INFO: `/v3/groups` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:53.575214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a36038-4d1f-42e0-85cf-356919c64849 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:53.576745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a36038-4d1f-42e0-85cf-356919c64849 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:53.593884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:53.596144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 167/324] 10.222.0.20 () {60 vars in 1134 bytes} [Tue Jul 12 03:35:53 2022] POST /identity/v3/groups => generated 226 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:35:53.648268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb6aac-9144-4b38-be91-f27e90a96a25 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:53.690088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3db43-e525-4292-bd00-8f4db0d2e952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:53.690910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3db43-e525-4292-bd00-8f4db0d2e952 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:53.691187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3db43-e525-4292-bd00-8f4db0d2e952 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:53.691515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3db43-e525-4292-bd00-8f4db0d2e952 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:53.704218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3db43-e525-4292-bd00-8f4db0d2e952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-c4f3db43-e525-4292-bd00-8f4db0d2e952 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:53.712106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:53.713088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:53.713874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 158/325] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:35:53.720080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a36038-4d1f-42e0-85cf-356919c64849 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:53.741466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14703aee-689e-4c4d-9d9c-160c3cf50264 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:53.741751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14703aee-689e-4c4d-9d9c-160c3cf50264 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:53.742030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14703aee-689e-4c4d-9d9c-160c3cf50264 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:53.743431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 159/326] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:35:53.749235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b5800-2065-4518-af0e-6906a1dc0634 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:53.749516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b5800-2065-4518-af0e-6906a1dc0634 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:53.749798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b5800-2065-4518-af0e-6906a1dc0634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:53.767178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-fa8b5800-2065-4518-af0e-6906a1dc0634 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:53.768412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-fa8b5800-2065-4518-af0e-6906a1dc0634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:53.827390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a36038-4d1f-42e0-85cf-356919c64849 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:53.854539 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 160/327] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:35:53.867767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982fa85-5b6b-4e0e-a66a-be35610963ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:53.867767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982fa85-5b6b-4e0e-a66a-be35610963ca None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:53.867767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982fa85-5b6b-4e0e-a66a-be35610963ca None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:53.867767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982fa85-5b6b-4e0e-a66a-be35610963ca None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:53.868164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982fa85-5b6b-4e0e-a66a-be35610963ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:53.868397 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982fa85-5b6b-4e0e-a66a-be35610963ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:53.869700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982fa85-5b6b-4e0e-a66a-be35610963ca None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:53.879462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 168/328] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:53.881294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b5800-2065-4518-af0e-6906a1dc0634 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:53.885528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982fa85-5b6b-4e0e-a66a-be35610963ca None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:53.938889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:53.948343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:54.006734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982fa85-5b6b-4e0e-a66a-be35610963ca None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:54.011821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b5800-2065-4518-af0e-6906a1dc0634 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:54.058428 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1b126-19cc-4ae9-9db8-9242b8471ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:54.058903 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd7f75-3787-45c9-9daa-2e7b12a7bc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:54.059251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1b126-19cc-4ae9-9db8-9242b8471ca0 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:54.059536 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1b126-19cc-4ae9-9db8-9242b8471ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:54.059812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1b126-19cc-4ae9-9db8-9242b8471ca0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:54.060732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd7f75-3787-45c9-9daa-2e7b12a7bc77 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:54.061214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd7f75-3787-45c9-9daa-2e7b12a7bc77 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:54.061685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd7f75-3787-45c9-9daa-2e7b12a7bc77 None admin] PATH_INFO: `/v3/users/f28344bd28f549f2bbc87b5d6b2ab5fd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:54.063126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd7f75-3787-45c9-9daa-2e7b12a7bc77 None admin] RBAC: Authorizing `identity:get_user(user_id=f28344bd28f549f2bbc87b5d6b2ab5fd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:54.064839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd7f75-3787-45c9-9daa-2e7b12a7bc77 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:54.067027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 169/329] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:53 2022] GET /identity/v3/users/f28344bd28f549f2bbc87b5d6b2ab5fd => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:54.071515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1b126-19cc-4ae9-9db8-9242b8471ca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:54.072679 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd7f75-3787-45c9-9daa-2e7b12a7bc77 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-dbd1b126-19cc-4ae9-9db8-9242b8471ca0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:54.080772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:54.081731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:54.082521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 161/330] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:54.090507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1b126-19cc-4ae9-9db8-9242b8471ca0 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:54.201586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd7f75-3787-45c9-9daa-2e7b12a7bc77 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:54.224693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1b126-19cc-4ae9-9db8-9242b8471ca0 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:54.234790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:54.238834 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:54.249788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d20035-dc27-475f-8258-0231239f3243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:54.250947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d20035-dc27-475f-8258-0231239f3243 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:54.251544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d20035-dc27-475f-8258-0231239f3243 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:54.251992 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d20035-dc27-475f-8258-0231239f3243 None admin] PATH_INFO: `/v3/system/users/f28344bd28f549f2bbc87b5d6b2ab5fd/roles/f740696dd1a847929b6236dcca34cc2f` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:54.254340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d20035-dc27-475f-8258-0231239f3243 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f28344bd28f549f2bbc87b5d6b2ab5fd, role_id=f740696dd1a847929b6236dcca34cc2f)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:54.256390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d20035-dc27-475f-8258-0231239f3243 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:54.268606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 170/331] 10.222.0.20 () {58 vars in 1334 bytes} [Tue Jul 12 03:35:54 2022] PUT /identity/v3/system/users/f28344bd28f549f2bbc87b5d6b2ab5fd/roles/f740696dd1a847929b6236dcca34cc2f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:35:54.278424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2764a-5c74-4b2b-9aa3-d463081a6acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:54.279426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2764a-5c74-4b2b-9aa3-d463081a6acc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:54.279923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2764a-5c74-4b2b-9aa3-d463081a6acc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:54.280358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2764a-5c74-4b2b-9aa3-d463081a6acc None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:54.281321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2764a-5c74-4b2b-9aa3-d463081a6acc None admin] RBAC: Adding query filter params (name=member) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:54.281765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2764a-5c74-4b2b-9aa3-d463081a6acc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:54.283522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2764a-5c74-4b2b-9aa3-d463081a6acc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:54.294973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 162/332] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:54.300998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d20035-dc27-475f-8258-0231239f3243 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:54.414989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d20035-dc27-475f-8258-0231239f3243 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:54.452991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02dd6b-0251-4959-a2b3-12c75ee3356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:54.454099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02dd6b-0251-4959-a2b3-12c75ee3356a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:54.454534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02dd6b-0251-4959-a2b3-12c75ee3356a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:54.454940 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02dd6b-0251-4959-a2b3-12c75ee3356a None admin] PATH_INFO: `/v3/users/836c5b1a1a33481693f7da6349a8dd29` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:54.456382 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02dd6b-0251-4959-a2b3-12c75ee3356a None admin] RBAC: Authorizing `identity:get_user(user_id=836c5b1a1a33481693f7da6349a8dd29)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:54.457980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02dd6b-0251-4959-a2b3-12c75ee3356a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:54.459763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 171/333] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:54 2022] GET /identity/v3/users/836c5b1a1a33481693f7da6349a8dd29 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:54.465648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2764a-5c74-4b2b-9aa3-d463081a6acc None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:54.585243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2764a-5c74-4b2b-9aa3-d463081a6acc None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:54.627491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c61bcf-1f30-46c9-9346-87badd6a3aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:54.628184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c61bcf-1f30-46c9-9346-87badd6a3aea None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:54.628466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c61bcf-1f30-46c9-9346-87badd6a3aea None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:54.628735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c61bcf-1f30-46c9-9346-87badd6a3aea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:54.629563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c61bcf-1f30-46c9-9346-87badd6a3aea None admin] RBAC: Adding query filter params (role.id=8f3ea95f038d4d8fbd1fd4897be36344, user.id=836c5b1a1a33481693f7da6349a8dd29, scope.system=all) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:54.629868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c61bcf-1f30-46c9-9346-87badd6a3aea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:54.631312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c61bcf-1f30-46c9-9346-87badd6a3aea None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:54.648306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 163/334] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Jul 12 03:35:54 2022] GET /identity/v3/role_assignments?user.id=836c5b1a1a33481693f7da6349a8dd29&scope.system=all&role.id=8f3ea95f038d4d8fbd1fd4897be36344 => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:55.956438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86845ce-2f7c-4dfa-a237-b388a8817ef0 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:55.956967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86845ce-2f7c-4dfa-a237-b388a8817ef0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:55.956967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86845ce-2f7c-4dfa-a237-b388a8817ef0 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:55.963766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 172/335] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:55.965061 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6bc6f-1b02-4198-8513-9620c552447f None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:55.965572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6bc6f-1b02-4198-8513-9620c552447f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:55.965764 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6bc6f-1b02-4198-8513-9620c552447f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:55.988126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-eee6bc6f-1b02-4198-8513-9620c552447f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:55.989306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-eee6bc6f-1b02-4198-8513-9620c552447f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:56.018961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:56.071063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 164/336] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:56.108646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86845ce-2f7c-4dfa-a237-b388a8817ef0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:56.166997 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:56.218383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86845ce-2f7c-4dfa-a237-b388a8817ef0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:56.254576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05884662-8878-42ba-96a0-756905e204ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:56.255386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05884662-8878-42ba-96a0-756905e204ab None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:56.255667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05884662-8878-42ba-96a0-756905e204ab None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:56.255943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05884662-8878-42ba-96a0-756905e204ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:56.267779 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05884662-8878-42ba-96a0-756905e204ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-05884662-8878-42ba-96a0-756905e204ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:56.275782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:56.276681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:56.277483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:56.277483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:56.277483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:56.277483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:56.277483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:56.277483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:56.277483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:56.277483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:56.277483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:56.277483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:56.277483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:56.277483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:56.278105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 173/337] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:56.283363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6bc6f-1b02-4198-8513-9620c552447f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:56.407346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6bc6f-1b02-4198-8513-9620c552447f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:56.450520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2c637-ddc3-4755-90a6-1fb8bc3e6675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:56.451200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2c637-ddc3-4755-90a6-1fb8bc3e6675 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:56.451479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2c637-ddc3-4755-90a6-1fb8bc3e6675 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:56.451749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2c637-ddc3-4755-90a6-1fb8bc3e6675 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:56.452532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2c637-ddc3-4755-90a6-1fb8bc3e6675 None admin] RBAC: Adding query filter params (name=member) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:56.452837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2c637-ddc3-4755-90a6-1fb8bc3e6675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:56.454327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2c637-ddc3-4755-90a6-1fb8bc3e6675 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:56.463085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 165/338] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:56.469044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05884662-8878-42ba-96a0-756905e204ab None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:56.583312 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05884662-8878-42ba-96a0-756905e204ab None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:56.597706 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:56.623822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285596d0-b252-4b5e-96b0-ed3c333e31b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:56.624519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285596d0-b252-4b5e-96b0-ed3c333e31b8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:56.624804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285596d0-b252-4b5e-96b0-ed3c333e31b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:56.625078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285596d0-b252-4b5e-96b0-ed3c333e31b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:56.707533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285596d0-b252-4b5e-96b0-ed3c333e31b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:56.709054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285596d0-b252-4b5e-96b0-ed3c333e31b8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:56.825140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 174/339] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:56.830860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2c637-ddc3-4755-90a6-1fb8bc3e6675 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:56.877183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:56.984827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e1f0a-a87a-4ec0-905d-6f319283bb4a None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:56.985332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e1f0a-a87a-4ec0-905d-6f319283bb4a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:56.985405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e1f0a-a87a-4ec0-905d-6f319283bb4a None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:56.987014 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 175/340] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:56.998828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd1ecd-ddcf-47d2-918c-69978a6a1b99 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:56.999112 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd1ecd-ddcf-47d2-918c-69978a6a1b99 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:56.999392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd1ecd-ddcf-47d2-918c-69978a6a1b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:56.999725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2c637-ddc3-4755-90a6-1fb8bc3e6675 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:57.024396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b5dd1ecd-ddcf-47d2-918c-69978a6a1b99 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:57.025608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b5dd1ecd-ddcf-47d2-918c-69978a6a1b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:57.041521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:57.085456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f07c85-b36a-4b1e-b891-13a71746c056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:57.086265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f07c85-b36a-4b1e-b891-13a71746c056 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:57.086552 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f07c85-b36a-4b1e-b891-13a71746c056 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:57.086832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f07c85-b36a-4b1e-b891-13a71746c056 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:57.092821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f07c85-b36a-4b1e-b891-13a71746c056 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:57.094355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f07c85-b36a-4b1e-b891-13a71746c056 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:57.096447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 166/341] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:56 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:57.101698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f07c85-b36a-4b1e-b891-13a71746c056 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:57.155665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 176/342] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:57.197648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd1ecd-ddcf-47d2-918c-69978a6a1b99 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:57.260910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:57.281075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f07c85-b36a-4b1e-b891-13a71746c056 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:57.322125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8a57d-6b97-450d-8d1e-0274ef5da950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:57.322997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8a57d-6b97-450d-8d1e-0274ef5da950 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:57.323281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8a57d-6b97-450d-8d1e-0274ef5da950 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:57.323561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8a57d-6b97-450d-8d1e-0274ef5da950 None admin] PATH_INFO: `/v3/groups/35044f9f91b24a7c98b9568355201b66` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:57.337068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8a57d-6b97-450d-8d1e-0274ef5da950 None admin] RBAC: Authorizing `identity:get_group(group_id=35044f9f91b24a7c98b9568355201b66)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:57.338500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8a57d-6b97-450d-8d1e-0274ef5da950 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:57.340423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 167/343] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:35:57 2022] GET /identity/v3/groups/35044f9f91b24a7c98b9568355201b66 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:35:57.341666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fce6c-4307-404c-bb8b-382437bca4f2 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:57.341955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fce6c-4307-404c-bb8b-382437bca4f2 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:57.342232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fce6c-4307-404c-bb8b-382437bca4f2 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:57.343468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 168/344] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:57.352972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fce6c-4307-404c-bb8b-382437bca4f2 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:57.422584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd1ecd-ddcf-47d2-918c-69978a6a1b99 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:57.492435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c6b02-6d72-4527-9a7e-b28d3c364f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:57.493692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c6b02-6d72-4527-9a7e-b28d3c364f73 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:57.494180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c6b02-6d72-4527-9a7e-b28d3c364f73 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:57.494637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c6b02-6d72-4527-9a7e-b28d3c364f73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:57.511072 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c6b02-6d72-4527-9a7e-b28d3c364f73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-2a9c6b02-6d72-4527-9a7e-b28d3c364f73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:57.520987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:57.521864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:57.522970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:57.522970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:57.522970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:57.522970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:57.522970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:57.522970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:57.522970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:57.522970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:57.522970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:57.522970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:57.522970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:57.522970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:57.523559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fce6c-4307-404c-bb8b-382437bca4f2 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:57.525793 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 177/345] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:35:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:57.527386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfb57c-845c-46ea-9e28-c473d447d934 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:57.527894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfb57c-845c-46ea-9e28-c473d447d934 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:57.528403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfb57c-845c-46ea-9e28-c473d447d934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:57.537788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:57.550562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b7cfb57c-845c-46ea-9e28-c473d447d934 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:57.552059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b7cfb57c-845c-46ea-9e28-c473d447d934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:57.564547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc2c93-1f13-4102-8296-8b58ea882cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:57.564871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc2c93-1f13-4102-8296-8b58ea882cc9 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:57.565100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc2c93-1f13-4102-8296-8b58ea882cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:57.565377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc2c93-1f13-4102-8296-8b58ea882cc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:57.566217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc2c93-1f13-4102-8296-8b58ea882cc9 None admin] RBAC: Adding query filter params (group.id=35044f9f91b24a7c98b9568355201b66, role.id=8f3ea95f038d4d8fbd1fd4897be36344, scope.project.id=b81760932eec4beda4343070aaf9b956) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:57.566531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc2c93-1f13-4102-8296-8b58ea882cc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:57.568096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc2c93-1f13-4102-8296-8b58ea882cc9 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:57.573717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 169/346] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 12 03:35:57 2022] GET /identity/v3/role_assignments?group.id=35044f9f91b24a7c98b9568355201b66&scope.project.id=b81760932eec4beda4343070aaf9b956&role.id=8f3ea95f038d4d8fbd1fd4897be36344 => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:57.575118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd79d8-f361-46f5-ae98-69e8778703b7 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:57.575411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd79d8-f361-46f5-ae98-69e8778703b7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:57.575691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd79d8-f361-46f5-ae98-69e8778703b7 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:57.577407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 170/347] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:57.578205 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd79d8-f361-46f5-ae98-69e8778703b7 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:57.596504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:57.640905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 178/348] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:57.642649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cef278-8132-4328-ac45-d9f481557b31 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:57.643159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cef278-8132-4328-ac45-d9f481557b31 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:57.643717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cef278-8132-4328-ac45-d9f481557b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:57.664155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-31cef278-8132-4328-ac45-d9f481557b31 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:57.665876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-31cef278-8132-4328-ac45-d9f481557b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:57.706202 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd79d8-f361-46f5-ae98-69e8778703b7 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:57.751664 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 179/349] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:57.752873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cef278-8132-4328-ac45-d9f481557b31 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:57.753916 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3a215-2cac-4578-9eae-8623e295a737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:57.759160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3a215-2cac-4578-9eae-8623e295a737 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:57.759160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3a215-2cac-4578-9eae-8623e295a737 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:57.759160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3a215-2cac-4578-9eae-8623e295a737 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:57.759160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3a215-2cac-4578-9eae-8623e295a737 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:57.759160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3a215-2cac-4578-9eae-8623e295a737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:57.759160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3a215-2cac-4578-9eae-8623e295a737 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:57.768780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 171/350] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:35:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:57.778418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3a215-2cac-4578-9eae-8623e295a737 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:57.864044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:57.888463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:57.904960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3a215-2cac-4578-9eae-8623e295a737 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:57.910003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cef278-8132-4328-ac45-d9f481557b31 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:57.958830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42db93-c5fa-4730-876e-d4d9b039e18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:57.959500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42db93-c5fa-4730-876e-d4d9b039e18c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:57.959779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42db93-c5fa-4730-876e-d4d9b039e18c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:57.960073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42db93-c5fa-4730-876e-d4d9b039e18c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:57.962962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927f551-77f2-4ef4-94b4-2ce021a87c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:57.963809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927f551-77f2-4ef4-94b4-2ce021a87c61 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:57.964113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927f551-77f2-4ef4-94b4-2ce021a87c61 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:57.964386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927f551-77f2-4ef4-94b4-2ce021a87c61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:57.989535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927f551-77f2-4ef4-94b4-2ce021a87c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a927f551-77f2-4ef4-94b4-2ce021a87c61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:58.003416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:58.004430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 180/351] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:58.005685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927f551-77f2-4ef4-94b4-2ce021a87c61 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:58.100795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42db93-c5fa-4730-876e-d4d9b039e18c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:58.102317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42db93-c5fa-4730-876e-d4d9b039e18c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:58.196110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927f551-77f2-4ef4-94b4-2ce021a87c61 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:58.200322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:58.232842 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:58.233502 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 172/352] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 03:35:58.234285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42db93-c5fa-4730-876e-d4d9b039e18c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:58.248587 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5154ab-51ce-4e06-8165-3379c72d4a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:58.249743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5154ab-51ce-4e06-8165-3379c72d4a04 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:58.250667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5154ab-51ce-4e06-8165-3379c72d4a04 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:58.251156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5154ab-51ce-4e06-8165-3379c72d4a04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:58.267656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5154ab-51ce-4e06-8165-3379c72d4a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-4c5154ab-51ce-4e06-8165-3379c72d4a04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:58.279519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:58.280498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:58.281330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:58.281330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:58.281330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:58.281330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:58.281330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:58.281330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:58.281330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:58.281330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:58.281330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:58.281330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:58.281330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:35:58.281330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:58.283924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 181/353] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:58.286188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6b862-5334-4372-9ea1-2c685ded25bb None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:58.286691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6b862-5334-4372-9ea1-2c685ded25bb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:58.287167 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6b862-5334-4372-9ea1-2c685ded25bb None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:58.292232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 182/354] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:58.293404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6b862-5334-4372-9ea1-2c685ded25bb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:58.371630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42db93-c5fa-4730-876e-d4d9b039e18c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:58.421985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762eda5-0422-4329-89cc-cb2d1f978362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:58.423278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762eda5-0422-4329-89cc-cb2d1f978362 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:58.424192 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762eda5-0422-4329-89cc-cb2d1f978362 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:58.424933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762eda5-0422-4329-89cc-cb2d1f978362 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:58.426136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762eda5-0422-4329-89cc-cb2d1f978362 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:58.427048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762eda5-0422-4329-89cc-cb2d1f978362 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:58.429201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762eda5-0422-4329-89cc-cb2d1f978362 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:58.433445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6b862-5334-4372-9ea1-2c685ded25bb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:58.440507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 173/355] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:58.441755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762eda5-0422-4329-89cc-cb2d1f978362 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:58.473559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9fb15-2b59-409b-b54a-61478d3ff8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:58.473559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9fb15-2b59-409b-b54a-61478d3ff8e8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:58.473559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9fb15-2b59-409b-b54a-61478d3ff8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:58.473901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9fb15-2b59-409b-b54a-61478d3ff8e8 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:58.475114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9fb15-2b59-409b-b54a-61478d3ff8e8 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:58.477258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9fb15-2b59-409b-b54a-61478d3ff8e8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:58.479589 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 183/356] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:58 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:58.481657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8579d-5e97-4e51-8f38-c9424abc6800 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:58.482050 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8579d-5e97-4e51-8f38-c9424abc6800 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:58.483111 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8579d-5e97-4e51-8f38-c9424abc6800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:58.493697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:58.506437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a0b8579d-5e97-4e51-8f38-c9424abc6800 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:58.507905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a0b8579d-5e97-4e51-8f38-c9424abc6800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:58.570822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:58.573350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762eda5-0422-4329-89cc-cb2d1f978362 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:58.601214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 184/357] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:58.602089 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8579d-5e97-4e51-8f38-c9424abc6800 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:58.626506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcc77a-9a7b-4b8a-8008-fe83a2afce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:58.627145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcc77a-9a7b-4b8a-8008-fe83a2afce4c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:58.627489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcc77a-9a7b-4b8a-8008-fe83a2afce4c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:58.627862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcc77a-9a7b-4b8a-8008-fe83a2afce4c None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:58.628889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcc77a-9a7b-4b8a-8008-fe83a2afce4c None admin] RBAC: Adding query filter params (name=member) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:58.629074 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcc77a-9a7b-4b8a-8008-fe83a2afce4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:58.630524 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcc77a-9a7b-4b8a-8008-fe83a2afce4c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:58.640164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 174/358] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:35:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:58.641253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447054ed-a0af-4dea-84e4-a3e561fb190e None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:58.641538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447054ed-a0af-4dea-84e4-a3e561fb190e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:58.641820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447054ed-a0af-4dea-84e4-a3e561fb190e None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:58.643301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 175/359] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:35:58.644046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447054ed-a0af-4dea-84e4-a3e561fb190e None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:58.734409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8579d-5e97-4e51-8f38-c9424abc6800 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:58.771123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:58.779558 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447054ed-a0af-4dea-84e4-a3e561fb190e None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:58.785677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee348c2-c9d4-4267-819c-2f655262b85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:58.786589 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee348c2-c9d4-4267-819c-2f655262b85d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:58.786872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee348c2-c9d4-4267-819c-2f655262b85d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:58.787146 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee348c2-c9d4-4267-819c-2f655262b85d None admin] PATH_INFO: `/v3/users/f28344bd28f549f2bbc87b5d6b2ab5fd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:58.788411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee348c2-c9d4-4267-819c-2f655262b85d None admin] RBAC: Authorizing `identity:get_user(user_id=f28344bd28f549f2bbc87b5d6b2ab5fd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:58.789924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee348c2-c9d4-4267-819c-2f655262b85d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:58.791612 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 185/360] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:35:58 2022] GET /identity/v3/users/f28344bd28f549f2bbc87b5d6b2ab5fd => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:58.792488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee348c2-c9d4-4267-819c-2f655262b85d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:58.824729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432599bc-5061-4ab1-8570-e6900da3ff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:58.825581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432599bc-5061-4ab1-8570-e6900da3ff77 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:58.825862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432599bc-5061-4ab1-8570-e6900da3ff77 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:58.826134 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432599bc-5061-4ab1-8570-e6900da3ff77 None admin] PATH_INFO: `/v3/groups/35044f9f91b24a7c98b9568355201b66` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:58.827314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432599bc-5061-4ab1-8570-e6900da3ff77 None admin] RBAC: Authorizing `identity:get_group(group_id=35044f9f91b24a7c98b9568355201b66)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:58.828861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432599bc-5061-4ab1-8570-e6900da3ff77 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:58.830645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 176/361] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:35:58 2022] GET /identity/v3/groups/35044f9f91b24a7c98b9568355201b66 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:58.831582 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8044efe-11b3-44e3-8ff2-47fcb101fe1d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:58.831871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8044efe-11b3-44e3-8ff2-47fcb101fe1d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:58.832174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8044efe-11b3-44e3-8ff2-47fcb101fe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:58.848587 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-b8044efe-11b3-44e3-8ff2-47fcb101fe1d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:35:58.849767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b8044efe-11b3-44e3-8ff2-47fcb101fe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:35:58.858270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:58.917986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee348c2-c9d4-4267-819c-2f655262b85d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:58.932124 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 177/362] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:35:58.933130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8044efe-11b3-44e3-8ff2-47fcb101fe1d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:58.965927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d69da0-c971-4668-ad06-d89ac48db6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:58.966776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d69da0-c971-4668-ad06-d89ac48db6d6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:58.967057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d69da0-c971-4668-ad06-d89ac48db6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:58.967326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d69da0-c971-4668-ad06-d89ac48db6d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:58.981725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d69da0-c971-4668-ad06-d89ac48db6d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c3d69da0-c971-4668-ad06-d89ac48db6d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:58.990009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:58.992140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 186/363] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:35:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:58.993156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d69da0-c971-4668-ad06-d89ac48db6d6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:59.067927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:59.073811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8044efe-11b3-44e3-8ff2-47fcb101fe1d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:59.113904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1383e-57e7-4179-9850-6babeec671d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:59.114544 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1383e-57e7-4179-9850-6babeec671d5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:59.114834 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1383e-57e7-4179-9850-6babeec671d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:59.115116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1383e-57e7-4179-9850-6babeec671d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:59.129968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d69da0-c971-4668-ad06-d89ac48db6d6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:59.147355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:59.180623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156d7e8-0e59-4cfb-9536-587c15a7638e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:59.181282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156d7e8-0e59-4cfb-9536-587c15a7638e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:59.181567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156d7e8-0e59-4cfb-9536-587c15a7638e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:59.182105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156d7e8-0e59-4cfb-9536-587c15a7638e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:59.182705 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156d7e8-0e59-4cfb-9536-587c15a7638e None admin] RBAC: Adding query filter params (role.id=f740696dd1a847929b6236dcca34cc2f, user.id=f28344bd28f549f2bbc87b5d6b2ab5fd, scope.system=all) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:59.183056 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156d7e8-0e59-4cfb-9536-587c15a7638e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:59.184673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156d7e8-0e59-4cfb-9536-587c15a7638e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:59.204602 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1383e-57e7-4179-9850-6babeec671d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:59.205814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1383e-57e7-4179-9850-6babeec671d5 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:59.209017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 187/364] 10.222.0.20 () {56 vars in 1308 bytes} [Tue Jul 12 03:35:58 2022] GET /identity/v3/role_assignments?user.id=f28344bd28f549f2bbc87b5d6b2ab5fd&scope.system=all&role.id=f740696dd1a847929b6236dcca34cc2f => generated 510 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:59.210335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156d7e8-0e59-4cfb-9536-587c15a7638e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:59.327374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:59.334510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 178/365] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:35:59.335739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1383e-57e7-4179-9850-6babeec671d5 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:59.356722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156d7e8-0e59-4cfb-9536-587c15a7638e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:59.400863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3b6ce-5b6a-4490-98c6-3939bbd7bb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:59.401738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3b6ce-5b6a-4490-98c6-3939bbd7bb7d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:59.402031 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3b6ce-5b6a-4490-98c6-3939bbd7bb7d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:59.402364 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3b6ce-5b6a-4490-98c6-3939bbd7bb7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:59.403552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3b6ce-5b6a-4490-98c6-3939bbd7bb7d None admin] RBAC: Adding query filter params (group.id=35044f9f91b24a7c98b9568355201b66, role.id=ff6066b1fb8940cd9ec82c078fd8d2f6, scope.project.id=b81760932eec4beda4343070aaf9b956) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:59.403872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3b6ce-5b6a-4490-98c6-3939bbd7bb7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:59.405915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3b6ce-5b6a-4490-98c6-3939bbd7bb7d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:59.413093 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 188/366] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 12 03:35:59 2022] GET /identity/v3/role_assignments?group.id=35044f9f91b24a7c98b9568355201b66&scope.project.id=b81760932eec4beda4343070aaf9b956&role.id=ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:59.414498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3b6ce-5b6a-4490-98c6-3939bbd7bb7d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:59.455985 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:59.472546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1383e-57e7-4179-9850-6babeec671d5 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:59.510700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981cbfd-1bfe-4b25-b780-2f82abbd1357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:59.511506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981cbfd-1bfe-4b25-b780-2f82abbd1357 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:59.511887 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981cbfd-1bfe-4b25-b780-2f82abbd1357 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:59.512283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981cbfd-1bfe-4b25-b780-2f82abbd1357 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:59.525669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981cbfd-1bfe-4b25-b780-2f82abbd1357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-c981cbfd-1bfe-4b25-b780-2f82abbd1357 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:35:59.533721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:35:59.534875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:35:59.535683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:35:59.535683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:35:59.535683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:35:59.535683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:35:59.535683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:35:59.535683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:35:59.535683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:35:59.535683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:35:59.535683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:35:59.535683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:35:59.535683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:35:59.535683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:35:59.536600 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 179/367] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:35:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:35:59.537638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981cbfd-1bfe-4b25-b780-2f82abbd1357 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:59.544895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3b6ce-5b6a-4490-98c6-3939bbd7bb7d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:59.582655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60a442-4c28-46c2-b56d-45cb154c4254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:59.583669 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60a442-4c28-46c2-b56d-45cb154c4254 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:59.584163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60a442-4c28-46c2-b56d-45cb154c4254 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:59.584717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60a442-4c28-46c2-b56d-45cb154c4254 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:59.585666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60a442-4c28-46c2-b56d-45cb154c4254 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:59.586143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60a442-4c28-46c2-b56d-45cb154c4254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:59.587831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60a442-4c28-46c2-b56d-45cb154c4254 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:59.597462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 189/368] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:35:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:59.598881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60a442-4c28-46c2-b56d-45cb154c4254 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:59.602849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:59.650773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981cbfd-1bfe-4b25-b780-2f82abbd1357 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:59.691627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c58f9-c392-48c8-a7d4-20fe04d7cb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:59.692532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c58f9-c392-48c8-a7d4-20fe04d7cb88 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:59.692809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c58f9-c392-48c8-a7d4-20fe04d7cb88 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:59.693468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c58f9-c392-48c8-a7d4-20fe04d7cb88 None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:59.694271 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c58f9-c392-48c8-a7d4-20fe04d7cb88 None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:59.696414 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c58f9-c392-48c8-a7d4-20fe04d7cb88 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:59.697866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 180/369] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:35:59 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:59.698690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c58f9-c392-48c8-a7d4-20fe04d7cb88 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:59.706446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:59.714861 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60a442-4c28-46c2-b56d-45cb154c4254 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:59.753569 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77c19e-b1e8-4465-82b1-60e9aa253193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:59.754254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77c19e-b1e8-4465-82b1-60e9aa253193 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:59.754535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77c19e-b1e8-4465-82b1-60e9aa253193 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:59.754835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77c19e-b1e8-4465-82b1-60e9aa253193 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:59.755627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77c19e-b1e8-4465-82b1-60e9aa253193 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:35:59.755935 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77c19e-b1e8-4465-82b1-60e9aa253193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:59.758637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77c19e-b1e8-4465-82b1-60e9aa253193 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:59.775433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 190/370] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:35:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:59.778147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77c19e-b1e8-4465-82b1-60e9aa253193 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:59.832211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c58f9-c392-48c8-a7d4-20fe04d7cb88 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:59.865572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:35:59.879073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946f57c-3de0-49c5-a998-49c5b5a8e4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:59.880146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946f57c-3de0-49c5-a998-49c5b5a8e4d8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:59.880633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946f57c-3de0-49c5-a998-49c5b5a8e4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:59.881106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946f57c-3de0-49c5-a998-49c5b5a8e4d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:59.902748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77c19e-b1e8-4465-82b1-60e9aa253193 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:35:59.950824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a2350-1779-4cf2-a549-b797d7608ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:35:59.950824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a2350-1779-4cf2-a549-b797d7608ab9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:35:59.950824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a2350-1779-4cf2-a549-b797d7608ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:35:59.951171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a2350-1779-4cf2-a549-b797d7608ab9 None admin] PATH_INFO: `/v3/groups/35044f9f91b24a7c98b9568355201b66` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:35:59.952809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a2350-1779-4cf2-a549-b797d7608ab9 None admin] RBAC: Authorizing `identity:get_group(group_id=35044f9f91b24a7c98b9568355201b66)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:59.954641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a2350-1779-4cf2-a549-b797d7608ab9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:59.959497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 191/371] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:35:59 2022] GET /identity/v3/groups/35044f9f91b24a7c98b9568355201b66 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:35:59.960730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a2350-1779-4cf2-a549-b797d7608ab9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:35:59.962070 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4946f57c-3de0-49c5-a998-49c5b5a8e4d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:35:59.964228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4946f57c-3de0-49c5-a998-49c5b5a8e4d8 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:35:59.991843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:00.089049 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a2350-1779-4cf2-a549-b797d7608ab9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:00.114722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 181/372] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:00.116677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e83a73-86bc-4625-a372-728811a649fc None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:00.116915 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e83a73-86bc-4625-a372-728811a649fc None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:00.117186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e83a73-86bc-4625-a372-728811a649fc None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:00.118928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 182/373] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:00.120104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e83a73-86bc-4625-a372-728811a649fc None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:00.136211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf095fb-3c6b-4c5b-b93c-739508768a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:00.137145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf095fb-3c6b-4c5b-b93c-739508768a2d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:00.137427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf095fb-3c6b-4c5b-b93c-739508768a2d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:00.137711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf095fb-3c6b-4c5b-b93c-739508768a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:00.152754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:00.227624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf095fb-3c6b-4c5b-b93c-739508768a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:00.229208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf095fb-3c6b-4c5b-b93c-739508768a2d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:00.253557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e83a73-86bc-4625-a372-728811a649fc None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:00.256158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:00.291067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0972f91-1bba-4fb5-b9b4-9f1b5c6b141c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:00.292335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0972f91-1bba-4fb5-b9b4-9f1b5c6b141c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:00.292755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0972f91-1bba-4fb5-b9b4-9f1b5c6b141c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:00.293290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0972f91-1bba-4fb5-b9b4-9f1b5c6b141c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:00.294609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0972f91-1bba-4fb5-b9b4-9f1b5c6b141c None admin] RBAC: Adding query filter params (group.id=35044f9f91b24a7c98b9568355201b66, role.id=8f3ea95f038d4d8fbd1fd4897be36344, scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:00.295296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0972f91-1bba-4fb5-b9b4-9f1b5c6b141c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:00.297351 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0972f91-1bba-4fb5-b9b4-9f1b5c6b141c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:00.304966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 183/374] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 12 03:36:00 2022] GET /identity/v3/role_assignments?group.id=35044f9f91b24a7c98b9568355201b66&scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62&role.id=8f3ea95f038d4d8fbd1fd4897be36344 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:00.305787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0972f91-1bba-4fb5-b9b4-9f1b5c6b141c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:00.362647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 192/375] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:35:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:00.367160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987c9fb-5dd7-4b6f-9da5-f6a61d7c8588 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:00.367927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987c9fb-5dd7-4b6f-9da5-f6a61d7c8588 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:00.368432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987c9fb-5dd7-4b6f-9da5-f6a61d7c8588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:00.396959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3987c9fb-5dd7-4b6f-9da5-f6a61d7c8588 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:00.398311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3987c9fb-5dd7-4b6f-9da5-f6a61d7c8588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:00.433046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:00.446635 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0972f91-1bba-4fb5-b9b4-9f1b5c6b141c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:00.491475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 193/376] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:00.494413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987c9fb-5dd7-4b6f-9da5-f6a61d7c8588 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:00.498980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f2b53-e744-4532-9bff-d0c65ba84d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:00.500070 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f2b53-e744-4532-9bff-d0c65ba84d5c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:00.500509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f2b53-e744-4532-9bff-d0c65ba84d5c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:00.501515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f2b53-e744-4532-9bff-d0c65ba84d5c None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:00.504233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f2b53-e744-4532-9bff-d0c65ba84d5c None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:00.505618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f2b53-e744-4532-9bff-d0c65ba84d5c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:00.507359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 184/377] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:36:00 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:00.512758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f2b53-e744-4532-9bff-d0c65ba84d5c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:00.565315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:00.611963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987c9fb-5dd7-4b6f-9da5-f6a61d7c8588 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:00.639661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f2b53-e744-4532-9bff-d0c65ba84d5c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:00.653163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26d5a5-dbc6-4ab4-9d03-3ff8615268ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:00.654041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26d5a5-dbc6-4ab4-9d03-3ff8615268ce None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:00.654317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26d5a5-dbc6-4ab4-9d03-3ff8615268ce None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:00.654591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26d5a5-dbc6-4ab4-9d03-3ff8615268ce None admin] PATH_INFO: `/v3/projects/579b7a08aae94d3fa59617c603862f30` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:00.655892 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26d5a5-dbc6-4ab4-9d03-3ff8615268ce None admin] RBAC: Authorizing `identity:get_project(project_id=579b7a08aae94d3fa59617c603862f30)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:00.657488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26d5a5-dbc6-4ab4-9d03-3ff8615268ce None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:00.659193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 194/378] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:36:00 2022] GET /identity/v3/projects/579b7a08aae94d3fa59617c603862f30 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:00.663970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26d5a5-dbc6-4ab4-9d03-3ff8615268ce None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:00.687425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492860da-6d2a-42da-b3b4-a70cfa4f59ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:00.688884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492860da-6d2a-42da-b3b4-a70cfa4f59ec None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:00.689363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492860da-6d2a-42da-b3b4-a70cfa4f59ec None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:00.689884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492860da-6d2a-42da-b3b4-a70cfa4f59ec None admin] PATH_INFO: `/v3/groups/35044f9f91b24a7c98b9568355201b66` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:00.691336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492860da-6d2a-42da-b3b4-a70cfa4f59ec None admin] RBAC: Authorizing `identity:get_group(group_id=35044f9f91b24a7c98b9568355201b66)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:00.693182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492860da-6d2a-42da-b3b4-a70cfa4f59ec None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:00.695515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 185/379] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:36:00 2022] GET /identity/v3/groups/35044f9f91b24a7c98b9568355201b66 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:00.696849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492860da-6d2a-42da-b3b4-a70cfa4f59ec None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:00.729343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:00.778755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26d5a5-dbc6-4ab4-9d03-3ff8615268ce None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:00.803980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:00.816905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d644fc-58ce-4659-be0b-79166e1039b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:00.817225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d644fc-58ce-4659-be0b-79166e1039b4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:00.817504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d644fc-58ce-4659-be0b-79166e1039b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:00.817773 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d644fc-58ce-4659-be0b-79166e1039b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:00.820733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492860da-6d2a-42da-b3b4-a70cfa4f59ec None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:00.828640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d644fc-58ce-4659-be0b-79166e1039b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-72d644fc-58ce-4659-be0b-79166e1039b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:00.836693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:00.837582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:00.838423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 195/380] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:00.839077 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d644fc-58ce-4659-be0b-79166e1039b4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:00.862289 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fde1ed-7159-45cb-8e9a-2e3095f32225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:00.863571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fde1ed-7159-45cb-8e9a-2e3095f32225 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:00.864100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fde1ed-7159-45cb-8e9a-2e3095f32225 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:00.864660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fde1ed-7159-45cb-8e9a-2e3095f32225 None admin] PATH_INFO: `/v3/groups/3d767ccc710b4546a820a8c65becf34b` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:00.871637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fde1ed-7159-45cb-8e9a-2e3095f32225 None admin] RBAC: Authorizing `identity:get_group(group_id=3d767ccc710b4546a820a8c65becf34b)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:00.873440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fde1ed-7159-45cb-8e9a-2e3095f32225 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:00.875493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 186/381] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:36:00 2022] GET /identity/v3/groups/3d767ccc710b4546a820a8c65becf34b => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:00.876681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fde1ed-7159-45cb-8e9a-2e3095f32225 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:00.945633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d644fc-58ce-4659-be0b-79166e1039b4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:00.985732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecffd8e-bbaa-4cfb-894e-ce35e465a071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:00.986392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecffd8e-bbaa-4cfb-894e-ce35e465a071 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:00.986673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecffd8e-bbaa-4cfb-894e-ce35e465a071 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:00.986948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecffd8e-bbaa-4cfb-894e-ce35e465a071 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:00.987778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecffd8e-bbaa-4cfb-894e-ce35e465a071 None admin] RBAC: Adding query filter params (group.id=35044f9f91b24a7c98b9568355201b66, role.id=ff6066b1fb8940cd9ec82c078fd8d2f6, scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:00.988126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecffd8e-bbaa-4cfb-894e-ce35e465a071 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:00.989571 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecffd8e-bbaa-4cfb-894e-ce35e465a071 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:00.996533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 196/382] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 12 03:36:00 2022] GET /identity/v3/role_assignments?group.id=35044f9f91b24a7c98b9568355201b66&scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62&role.id=ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:00.997371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecffd8e-bbaa-4cfb-894e-ce35e465a071 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:01.001364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:01.011827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fde1ed-7159-45cb-8e9a-2e3095f32225 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:01.053211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e4d93-40fe-4d5e-ba4c-e5244ce078da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:01.054029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e4d93-40fe-4d5e-ba4c-e5244ce078da None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:01.054312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e4d93-40fe-4d5e-ba4c-e5244ce078da None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:01.054588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e4d93-40fe-4d5e-ba4c-e5244ce078da None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:01.055398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e4d93-40fe-4d5e-ba4c-e5244ce078da None admin] RBAC: Adding query filter params (name=member) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:01.055713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e4d93-40fe-4d5e-ba4c-e5244ce078da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:01.057207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e4d93-40fe-4d5e-ba4c-e5244ce078da None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:01.067249 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 187/383] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:36:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:01.073083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:01.073989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e4d93-40fe-4d5e-ba4c-e5244ce078da None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:01.131590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecffd8e-bbaa-4cfb-894e-ce35e465a071 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:01.169428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9c73f-3ca4-44e7-ac0d-488f8f4a7ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:01.170102 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9c73f-3ca4-44e7-ac0d-488f8f4a7ce0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:01.170395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9c73f-3ca4-44e7-ac0d-488f8f4a7ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:01.170653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9c73f-3ca4-44e7-ac0d-488f8f4a7ce0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:01.171511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9c73f-3ca4-44e7-ac0d-488f8f4a7ce0 None admin] RBAC: Adding query filter params (group.id=3d767ccc710b4546a820a8c65becf34b, role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=579b7a08aae94d3fa59617c603862f30) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:01.171820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9c73f-3ca4-44e7-ac0d-488f8f4a7ce0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:01.173317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9c73f-3ca4-44e7-ac0d-488f8f4a7ce0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:01.179972 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 197/384] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 12 03:36:00 2022] GET /identity/v3/role_assignments?group.id=3d767ccc710b4546a820a8c65becf34b&scope.project.id=579b7a08aae94d3fa59617c603862f30&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:01.256659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e4d93-40fe-4d5e-ba4c-e5244ce078da None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:01.303738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a142fb-3cb3-4399-be66-27129ed60630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:01.304671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a142fb-3cb3-4399-be66-27129ed60630 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:01.304954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a142fb-3cb3-4399-be66-27129ed60630 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:01.305232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a142fb-3cb3-4399-be66-27129ed60630 None admin] PATH_INFO: `/v3/groups/35044f9f91b24a7c98b9568355201b66` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:01.309033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a142fb-3cb3-4399-be66-27129ed60630 None admin] RBAC: Authorizing `identity:get_group(group_id=35044f9f91b24a7c98b9568355201b66)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:01.310532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a142fb-3cb3-4399-be66-27129ed60630 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:01.312650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 188/385] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:36:01 2022] GET /identity/v3/groups/35044f9f91b24a7c98b9568355201b66 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:01.318493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9c73f-3ca4-44e7-ac0d-488f8f4a7ce0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:01.441358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9c73f-3ca4-44e7-ac0d-488f8f4a7ce0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:01.452239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:01.477092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920e6af-4275-4099-ac3e-1b053e4d2824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:01.477961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920e6af-4275-4099-ac3e-1b053e4d2824 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:01.478238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920e6af-4275-4099-ac3e-1b053e4d2824 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:01.478510 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920e6af-4275-4099-ac3e-1b053e4d2824 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:01.479753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920e6af-4275-4099-ac3e-1b053e4d2824 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:01.481226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920e6af-4275-4099-ac3e-1b053e4d2824 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:01.483256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 198/386] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:36:01 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:01.488632 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a142fb-3cb3-4399-be66-27129ed60630 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:01.533689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:01.642087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a142fb-3cb3-4399-be66-27129ed60630 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:01.688699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97b3c0-94d3-412c-95c6-98116206ff5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:01.689851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97b3c0-94d3-412c-95c6-98116206ff5a None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:01.690300 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97b3c0-94d3-412c-95c6-98116206ff5a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:01.690727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97b3c0-94d3-412c-95c6-98116206ff5a None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956/groups/35044f9f91b24a7c98b9568355201b66/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:01.692877 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97b3c0-94d3-412c-95c6-98116206ff5a None admin] RBAC: Authorizing `identity:create_grant(project_id=b81760932eec4beda4343070aaf9b956, group_id=35044f9f91b24a7c98b9568355201b66, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:01.694473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97b3c0-94d3-412c-95c6-98116206ff5a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:01.717770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 189/387] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Jul 12 03:36:01 2022] PUT /identity/v3/projects/b81760932eec4beda4343070aaf9b956/groups/35044f9f91b24a7c98b9568355201b66/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:36:01.834823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b6cc9-31c7-4bd4-a8ff-b7aff4ad8c50 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:01.835278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b6cc9-31c7-4bd4-a8ff-b7aff4ad8c50 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:01.835649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b6cc9-31c7-4bd4-a8ff-b7aff4ad8c50 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:01.837478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 199/388] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:01.844060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb5e1-cd8c-4f95-8585-79f4d978d54b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:01.844682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb5e1-cd8c-4f95-8585-79f4d978d54b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:01.845164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bb5e1-cd8c-4f95-8585-79f4d978d54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:01.869163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-fa2bb5e1-cd8c-4f95-8585-79f4d978d54b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:01.870847 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-fa2bb5e1-cd8c-4f95-8585-79f4d978d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:01.948544 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 190/389] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:01.976067 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b6cc9-31c7-4bd4-a8ff-b7aff4ad8c50 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:02.081626 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b6cc9-31c7-4bd4-a8ff-b7aff4ad8c50 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:02.121376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe2c59-bf99-4d1a-b897-482d07f43743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:02.122135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe2c59-bf99-4d1a-b897-482d07f43743 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:02.122415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe2c59-bf99-4d1a-b897-482d07f43743 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:02.122687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe2c59-bf99-4d1a-b897-482d07f43743 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:02.138195 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe2c59-bf99-4d1a-b897-482d07f43743 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-77fe2c59-bf99-4d1a-b897-482d07f43743 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:02.147040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:02.147925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:02.148763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:02.148763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:02.148763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:02.148763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:02.148763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:02.148763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:02.148763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:02.148763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:02.148763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:02.148763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:02.148763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:36:02.148763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:02.156456 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 200/390] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:02.165132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bb5e1-cd8c-4f95-8585-79f4d978d54b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:02.181191 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:02.311674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bb5e1-cd8c-4f95-8585-79f4d978d54b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:02.353723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f236d4-718f-462c-9682-41ad4593e95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:02.354391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f236d4-718f-462c-9682-41ad4593e95e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:02.354675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f236d4-718f-462c-9682-41ad4593e95e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:02.354945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f236d4-718f-462c-9682-41ad4593e95e None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:02.355692 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f236d4-718f-462c-9682-41ad4593e95e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:02.355995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f236d4-718f-462c-9682-41ad4593e95e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:02.357385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f236d4-718f-462c-9682-41ad4593e95e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:02.369835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 191/391] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:02.375398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe2c59-bf99-4d1a-b897-482d07f43743 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:02.418506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:02.480439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe2c59-bf99-4d1a-b897-482d07f43743 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:02.514027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e6324-de8e-4a0a-82b7-1742f6857450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:02.514885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e6324-de8e-4a0a-82b7-1742f6857450 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:02.515160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e6324-de8e-4a0a-82b7-1742f6857450 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:02.515430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e6324-de8e-4a0a-82b7-1742f6857450 None admin] PATH_INFO: `/v3/groups/35044f9f91b24a7c98b9568355201b66` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:02.516629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e6324-de8e-4a0a-82b7-1742f6857450 None admin] RBAC: Authorizing `identity:get_group(group_id=35044f9f91b24a7c98b9568355201b66)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:02.518125 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e6324-de8e-4a0a-82b7-1742f6857450 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:02.520089 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 201/392] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:36:02 2022] GET /identity/v3/groups/35044f9f91b24a7c98b9568355201b66 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:02.526141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f236d4-718f-462c-9682-41ad4593e95e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:02.613049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:02.647938 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f236d4-718f-462c-9682-41ad4593e95e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:02.684027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b38bcb-d65c-4e57-9df5-da96dda667c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:02.684908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38bcb-d65c-4e57-9df5-da96dda667c5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:02.685188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38bcb-d65c-4e57-9df5-da96dda667c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:02.685459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38bcb-d65c-4e57-9df5-da96dda667c5 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:02.686644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b38bcb-d65c-4e57-9df5-da96dda667c5 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:02.688163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b38bcb-d65c-4e57-9df5-da96dda667c5 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:02.689985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 192/393] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:36:02 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:02.695693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e6324-de8e-4a0a-82b7-1742f6857450 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:02.805204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e6324-de8e-4a0a-82b7-1742f6857450 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:02.812346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:02.847683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c89b4-1bfe-4b39-a2f9-3c44c1b1f894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:02.848601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c89b4-1bfe-4b39-a2f9-3c44c1b1f894 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:02.848883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c89b4-1bfe-4b39-a2f9-3c44c1b1f894 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:02.849157 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c89b4-1bfe-4b39-a2f9-3c44c1b1f894 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956/groups/35044f9f91b24a7c98b9568355201b66/roles/ff6066b1fb8940cd9ec82c078fd8d2f6` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:02.851354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c89b4-1bfe-4b39-a2f9-3c44c1b1f894 None admin] RBAC: Authorizing `identity:create_grant(project_id=b81760932eec4beda4343070aaf9b956, group_id=35044f9f91b24a7c98b9568355201b66, role_id=ff6066b1fb8940cd9ec82c078fd8d2f6)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:02.852822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c89b4-1bfe-4b39-a2f9-3c44c1b1f894 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:02.867585 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 202/394] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Jul 12 03:36:02 2022] PUT /identity/v3/projects/b81760932eec4beda4343070aaf9b956/groups/35044f9f91b24a7c98b9568355201b66/roles/ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:36:03.322717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d39a2-26a1-4406-a7a6-9cdf96f802a8 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:03.323260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d39a2-26a1-4406-a7a6-9cdf96f802a8 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:03.323260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d39a2-26a1-4406-a7a6-9cdf96f802a8 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:03.324660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 193/395] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:03.329990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07847b-822e-4aea-8364-88cc9025cc5f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:03.330491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07847b-822e-4aea-8364-88cc9025cc5f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:03.330652 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07847b-822e-4aea-8364-88cc9025cc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:03.357087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2b07847b-822e-4aea-8364-88cc9025cc5f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:03.358294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2b07847b-822e-4aea-8364-88cc9025cc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:03.450547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 203/396] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:03.487152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d39a2-26a1-4406-a7a6-9cdf96f802a8 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:03.569249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc4e5f-af65-4277-8735-21d75d184245 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:03.569610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc4e5f-af65-4277-8735-21d75d184245 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:03.569830 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc4e5f-af65-4277-8735-21d75d184245 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:03.571442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 204/397] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:03.576585 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db810853-c335-471c-bd00-ee93166904a8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:03.576793 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db810853-c335-471c-bd00-ee93166904a8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:03.577066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db810853-c335-471c-bd00-ee93166904a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:03.601469 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-db810853-c335-471c-bd00-ee93166904a8 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:03.602652 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-db810853-c335-471c-bd00-ee93166904a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:03.629364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d39a2-26a1-4406-a7a6-9cdf96f802a8 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:03.667482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8500e0-79cd-444c-91eb-4797fc1a3c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:03.668329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8500e0-79cd-444c-91eb-4797fc1a3c5f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:03.668894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8500e0-79cd-444c-91eb-4797fc1a3c5f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:03.669223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8500e0-79cd-444c-91eb-4797fc1a3c5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:03.680136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:03.685313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8500e0-79cd-444c-91eb-4797fc1a3c5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:03.692142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 205/398] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-6b8500e0-79cd-444c-91eb-4797fc1a3c5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:03.693155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:03.694001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:03.694844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:03.694844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:03.694844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:03.694844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:03.694844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:03.694844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:03.694844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:03.694844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:03.694844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:03.694844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:03.694844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:36:03.694844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:03.695447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dada5-e857-4160-98e4-07aa83ddc992 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:03.695447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dada5-e857-4160-98e4-07aa83ddc992 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:03.695447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dada5-e857-4160-98e4-07aa83ddc992 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:03.695629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 194/399] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:03.695694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 206/400] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:03.700741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8dada5-e857-4160-98e4-07aa83ddc992 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:03.701142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e231d1-3510-4b97-b906-8cbf04ba4d4f None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:03.701406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e231d1-3510-4b97-b906-8cbf04ba4d4f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:03.701689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e231d1-3510-4b97-b906-8cbf04ba4d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:03.721249 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f6e231d1-3510-4b97-b906-8cbf04ba4d4f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:03.723107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f6e231d1-3510-4b97-b906-8cbf04ba4d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:03.745775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:03.820117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 195/401] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:03.820998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e231d1-3510-4b97-b906-8cbf04ba4d4f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:03.847737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8dada5-e857-4160-98e4-07aa83ddc992 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:03.892520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9174bd-890c-4865-8556-49f0e0effcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:03.893220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9174bd-890c-4865-8556-49f0e0effcfb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:03.894150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9174bd-890c-4865-8556-49f0e0effcfb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:03.894428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9174bd-890c-4865-8556-49f0e0effcfb None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:03.895177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9174bd-890c-4865-8556-49f0e0effcfb None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:03.895834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9174bd-890c-4865-8556-49f0e0effcfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:03.897281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9174bd-890c-4865-8556-49f0e0effcfb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:03.913138 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 207/402] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:36:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:03.914384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9174bd-890c-4865-8556-49f0e0effcfb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:03.943445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e231d1-3510-4b97-b906-8cbf04ba4d4f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:03.979362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:03.987986 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981e5a2-0987-49c8-ba76-768e5cfd6ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:03.989061 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981e5a2-0987-49c8-ba76-768e5cfd6ddc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:03.989347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981e5a2-0987-49c8-ba76-768e5cfd6ddc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:03.989621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981e5a2-0987-49c8-ba76-768e5cfd6ddc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.004278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981e5a2-0987-49c8-ba76-768e5cfd6ddc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-d981e5a2-0987-49c8-ba76-768e5cfd6ddc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:04.013573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:04.014464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 196/403] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:04.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981e5a2-0987-49c8-ba76-768e5cfd6ddc None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:04.036538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:04.051931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9174bd-890c-4865-8556-49f0e0effcfb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:04.094067 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae29d0-7f2d-433b-b627-52c0f90b05e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:04.095695 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae29d0-7f2d-433b-b627-52c0f90b05e3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:04.096529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae29d0-7f2d-433b-b627-52c0f90b05e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:04.097007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae29d0-7f2d-433b-b627-52c0f90b05e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.110614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae29d0-7f2d-433b-b627-52c0f90b05e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-06ae29d0-7f2d-433b-b627-52c0f90b05e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:04.123325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:04.124385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:04.125367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:04.125367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:04.125367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:04.125367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:04.125367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:04.125367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:04.125367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:04.125367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:04.125367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:04.125367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:04.125367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:36:04.125367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:04.129275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 208/404] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:04.130542 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae29d0-7f2d-433b-b627-52c0f90b05e3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:04.140073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981e5a2-0987-49c8-ba76-768e5cfd6ddc None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:04.176956 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8467c-8ffb-4420-8032-326081847294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:04.177992 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8467c-8ffb-4420-8032-326081847294 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:04.178379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8467c-8ffb-4420-8032-326081847294 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:04.178749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8467c-8ffb-4420-8032-326081847294 None admin] PATH_INFO: `/v3/groups/35044f9f91b24a7c98b9568355201b66` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.180070 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8467c-8ffb-4420-8032-326081847294 None admin] RBAC: Authorizing `identity:get_group(group_id=35044f9f91b24a7c98b9568355201b66)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:04.182247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8467c-8ffb-4420-8032-326081847294 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:04.188997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 197/405] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:36:04 2022] GET /identity/v3/groups/35044f9f91b24a7c98b9568355201b66 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:04.188997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8467c-8ffb-4420-8032-326081847294 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:04.259148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae29d0-7f2d-433b-b627-52c0f90b05e3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:04.263834 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:04.296775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15003bb7-9e55-4f37-8e8d-f2c24b958c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:04.297727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15003bb7-9e55-4f37-8e8d-f2c24b958c3c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:04.298228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15003bb7-9e55-4f37-8e8d-f2c24b958c3c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:04.298618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15003bb7-9e55-4f37-8e8d-f2c24b958c3c None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.299459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15003bb7-9e55-4f37-8e8d-f2c24b958c3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:04.300133 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15003bb7-9e55-4f37-8e8d-f2c24b958c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:04.301981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15003bb7-9e55-4f37-8e8d-f2c24b958c3c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:04.316095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 209/406] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:04.316095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15003bb7-9e55-4f37-8e8d-f2c24b958c3c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:04.325433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8467c-8ffb-4420-8032-326081847294 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:04.329508 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:04.372409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47317bb-6624-42c8-962d-d197746eb19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:04.373526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47317bb-6624-42c8-962d-d197746eb19c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:04.374013 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47317bb-6624-42c8-962d-d197746eb19c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:04.374498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47317bb-6624-42c8-962d-d197746eb19c None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.375455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47317bb-6624-42c8-962d-d197746eb19c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:04.375954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47317bb-6624-42c8-962d-d197746eb19c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:04.377666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47317bb-6624-42c8-962d-d197746eb19c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:04.393662 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 198/407] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:04.395570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47317bb-6624-42c8-962d-d197746eb19c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:04.442674 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15003bb7-9e55-4f37-8e8d-f2c24b958c3c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:04.477912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3edf5-14cb-4958-9f67-373ceeda46ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:04.478781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3edf5-14cb-4958-9f67-373ceeda46ae None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:04.479065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3edf5-14cb-4958-9f67-373ceeda46ae None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:04.479348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3edf5-14cb-4958-9f67-373ceeda46ae None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.480565 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3edf5-14cb-4958-9f67-373ceeda46ae None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:04.482027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3edf5-14cb-4958-9f67-373ceeda46ae None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:04.483720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 210/408] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:36:04 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:04.484500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3edf5-14cb-4958-9f67-373ceeda46ae None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:04.514593 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47317bb-6624-42c8-962d-d197746eb19c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:04.539493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:04.556954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466af69d-3d5e-47cb-bae3-dc3d27f70a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:04.558038 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466af69d-3d5e-47cb-bae3-dc3d27f70a63 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:04.558516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466af69d-3d5e-47cb-bae3-dc3d27f70a63 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:04.559335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466af69d-3d5e-47cb-bae3-dc3d27f70a63 None admin] PATH_INFO: `/v3/groups/3d767ccc710b4546a820a8c65becf34b` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.561055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466af69d-3d5e-47cb-bae3-dc3d27f70a63 None admin] RBAC: Authorizing `identity:get_group(group_id=3d767ccc710b4546a820a8c65becf34b)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:04.562328 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:04.563157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466af69d-3d5e-47cb-bae3-dc3d27f70a63 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:04.565566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 199/409] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:36:04 2022] GET /identity/v3/groups/3d767ccc710b4546a820a8c65becf34b => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:04.567591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2a028-5a89-4706-91bf-5d9330fbb5f5 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:04.568097 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2a028-5a89-4706-91bf-5d9330fbb5f5 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:04.568651 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2a028-5a89-4706-91bf-5d9330fbb5f5 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.570702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 200/410] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:04.571627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2a028-5a89-4706-91bf-5d9330fbb5f5 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:04.583927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3edf5-14cb-4958-9f67-373ceeda46ae None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:04.619371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59086a21-3fc7-44aa-87cf-c18bfbbb9742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:04.620647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086a21-3fc7-44aa-87cf-c18bfbbb9742 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:04.620932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086a21-3fc7-44aa-87cf-c18bfbbb9742 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:04.621206 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086a21-3fc7-44aa-87cf-c18bfbbb9742 None admin] PATH_INFO: `/v3/groups/35044f9f91b24a7c98b9568355201b66` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.622769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59086a21-3fc7-44aa-87cf-c18bfbbb9742 None admin] RBAC: Authorizing `identity:get_group(group_id=35044f9f91b24a7c98b9568355201b66)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:04.624533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59086a21-3fc7-44aa-87cf-c18bfbbb9742 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:04.630957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 211/411] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:36:04 2022] GET /identity/v3/groups/35044f9f91b24a7c98b9568355201b66 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:04.630957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59086a21-3fc7-44aa-87cf-c18bfbbb9742 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:04.689839 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2a028-5a89-4706-91bf-5d9330fbb5f5 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:04.724927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d54c20-b008-41d3-bc8a-75498b74641b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:04.725804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d54c20-b008-41d3-bc8a-75498b74641b None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:04.726086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d54c20-b008-41d3-bc8a-75498b74641b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:04.726361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d54c20-b008-41d3-bc8a-75498b74641b None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62/groups/35044f9f91b24a7c98b9568355201b66/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.728419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d54c20-b008-41d3-bc8a-75498b74641b None admin] RBAC: Authorizing `identity:create_grant(project_id=3bcbe8d7aaa3411b844ce5c42db14e62, group_id=35044f9f91b24a7c98b9568355201b66, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:04.729883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d54c20-b008-41d3-bc8a-75498b74641b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:04.738046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59086a21-3fc7-44aa-87cf-c18bfbbb9742 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:04.756037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 201/412] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Jul 12 03:36:04 2022] PUT /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62/groups/35044f9f91b24a7c98b9568355201b66/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:36:04.756037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee670540-b661-47bc-b4bc-431d1d8a217e None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:04.756037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee670540-b661-47bc-b4bc-431d1d8a217e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:04.756037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee670540-b661-47bc-b4bc-431d1d8a217e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.776378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d989e9b-8a0f-4d0e-ac5a-ff950990a2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:04.777276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d989e9b-8a0f-4d0e-ac5a-ff950990a2b6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:04.777568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d989e9b-8a0f-4d0e-ac5a-ff950990a2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:04.777947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-ee670540-b661-47bc-b4bc-431d1d8a217e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:04.778085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d989e9b-8a0f-4d0e-ac5a-ff950990a2b6 None admin] PATH_INFO: `/v3/projects/579b7a08aae94d3fa59617c603862f30` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.779507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ee670540-b661-47bc-b4bc-431d1d8a217e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:04.779720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d989e9b-8a0f-4d0e-ac5a-ff950990a2b6 None admin] RBAC: Authorizing `identity:get_project(project_id=579b7a08aae94d3fa59617c603862f30)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:04.781367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d989e9b-8a0f-4d0e-ac5a-ff950990a2b6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:04.784976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 212/413] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:36:04 2022] GET /identity/v3/projects/579b7a08aae94d3fa59617c603862f30 => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:04.785847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d989e9b-8a0f-4d0e-ac5a-ff950990a2b6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:04.807860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:04.848434 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:04.875141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 202/414] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:04.876736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee670540-b661-47bc-b4bc-431d1d8a217e None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:04.934640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d989e9b-8a0f-4d0e-ac5a-ff950990a2b6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:04.994302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7059e-b1f1-437d-8696-e8f6028db31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:04.995434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7059e-b1f1-437d-8696-e8f6028db31c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:04.995722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7059e-b1f1-437d-8696-e8f6028db31c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:04.996050 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7059e-b1f1-437d-8696-e8f6028db31c None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:04.997651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7059e-b1f1-437d-8696-e8f6028db31c None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:04.999119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7059e-b1f1-437d-8696-e8f6028db31c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:05.001409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 213/415] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:36:04 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:05.002313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7059e-b1f1-437d-8696-e8f6028db31c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:05.014433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee670540-b661-47bc-b4bc-431d1d8a217e None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:05.061245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633c0fd-444a-4212-8a47-3a4dd1cf75f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:05.062138 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633c0fd-444a-4212-8a47-3a4dd1cf75f6 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:05.062423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633c0fd-444a-4212-8a47-3a4dd1cf75f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:05.062700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633c0fd-444a-4212-8a47-3a4dd1cf75f6 None admin] PATH_INFO: `/v3/projects/579b7a08aae94d3fa59617c603862f30/groups/3d767ccc710b4546a820a8c65becf34b/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:05.065622 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633c0fd-444a-4212-8a47-3a4dd1cf75f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=579b7a08aae94d3fa59617c603862f30, group_id=3d767ccc710b4546a820a8c65becf34b, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:05.067766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633c0fd-444a-4212-8a47-3a4dd1cf75f6 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:05.085290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 203/416] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Jul 12 03:36:04 2022] PUT /identity/v3/projects/579b7a08aae94d3fa59617c603862f30/groups/3d767ccc710b4546a820a8c65becf34b/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:36:05.092209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633c0fd-444a-4212-8a47-3a4dd1cf75f6 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:05.097060 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:05.126360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7059e-b1f1-437d-8696-e8f6028db31c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:05.161485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:05.175242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b403b-3b03-4e47-aeed-3990ade4dbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:05.176961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b403b-3b03-4e47-aeed-3990ade4dbe2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:05.177973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b403b-3b03-4e47-aeed-3990ade4dbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:05.178625 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b403b-3b03-4e47-aeed-3990ade4dbe2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:05.207633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b403b-3b03-4e47-aeed-3990ade4dbe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-bf9b403b-3b03-4e47-aeed-3990ade4dbe2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:05.221785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:05.222780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:05.224068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 214/417] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:05.229826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b403b-3b03-4e47-aeed-3990ade4dbe2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:05.298867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633c0fd-444a-4212-8a47-3a4dd1cf75f6 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:05.335043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe597f-700a-4a05-b71f-c8f06d3da2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:05.335924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe597f-700a-4a05-b71f-c8f06d3da2a1 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:05.336528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe597f-700a-4a05-b71f-c8f06d3da2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:05.336528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe597f-700a-4a05-b71f-c8f06d3da2a1 None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62/groups/35044f9f91b24a7c98b9568355201b66/roles/ff6066b1fb8940cd9ec82c078fd8d2f6` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:05.338729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe597f-700a-4a05-b71f-c8f06d3da2a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3bcbe8d7aaa3411b844ce5c42db14e62, group_id=35044f9f91b24a7c98b9568355201b66, role_id=ff6066b1fb8940cd9ec82c078fd8d2f6)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:05.340353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe597f-700a-4a05-b71f-c8f06d3da2a1 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:05.358175 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 204/418] 10.222.0.20 () {58 vars in 1442 bytes} [Tue Jul 12 03:36:05 2022] PUT /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62/groups/35044f9f91b24a7c98b9568355201b66/roles/ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:36:05.358967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b403b-3b03-4e47-aeed-3990ade4dbe2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:05.359513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8348556-e639-4f7a-a100-bcef06de91c7 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:05.359800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8348556-e639-4f7a-a100-bcef06de91c7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:05.360092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8348556-e639-4f7a-a100-bcef06de91c7 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:05.361709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 205/419] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:05.367185 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b5a6f-e894-4079-9c28-540827c81ced None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:05.367499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b5a6f-e894-4079-9c28-540827c81ced None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:05.367747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b5a6f-e894-4079-9c28-540827c81ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:05.388281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-342b5a6f-e894-4079-9c28-540827c81ced None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:05.389559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-342b5a6f-e894-4079-9c28-540827c81ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:05.404061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23fb0d-d170-4e0f-bf51-b53439a8c8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:05.405190 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23fb0d-d170-4e0f-bf51-b53439a8c8d5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:05.405693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23fb0d-d170-4e0f-bf51-b53439a8c8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:05.406157 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23fb0d-d170-4e0f-bf51-b53439a8c8d5 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:05.407199 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23fb0d-d170-4e0f-bf51-b53439a8c8d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:05.407698 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23fb0d-d170-4e0f-bf51-b53439a8c8d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:05.409431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23fb0d-d170-4e0f-bf51-b53439a8c8d5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:05.419090 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 215/420] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:36:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:05.424864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23fb0d-d170-4e0f-bf51-b53439a8c8d5 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:05.430751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:05.474638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 206/421] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:36:05.498547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b5a6f-e894-4079-9c28-540827c81ced None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:05.529059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:05.553825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23fb0d-d170-4e0f-bf51-b53439a8c8d5 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:05.591929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809353eb-5758-4be3-9264-f79e51e482af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:05.592945 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809353eb-5758-4be3-9264-f79e51e482af None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:05.593415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809353eb-5758-4be3-9264-f79e51e482af None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:05.593872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809353eb-5758-4be3-9264-f79e51e482af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:05.616179 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b5a6f-e894-4079-9c28-540827c81ced None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:05.662527 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50152b-f8c1-4d36-84fc-528b0ac4dff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:05.663968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50152b-f8c1-4d36-84fc-528b0ac4dff9 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:05.664367 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50152b-f8c1-4d36-84fc-528b0ac4dff9 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:05.665000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50152b-f8c1-4d36-84fc-528b0ac4dff9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:05.684064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809353eb-5758-4be3-9264-f79e51e482af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:05.684247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50152b-f8c1-4d36-84fc-528b0ac4dff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:05.686072 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809353eb-5758-4be3-9264-f79e51e482af None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-ea50152b-f8c1-4d36-84fc-528b0ac4dff9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:05.687190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:05.688064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:05.688835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:05.688835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:05.688835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:05.688835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:05.688835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:05.688835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:05.688835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:05.688835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:05.688835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:05.688835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:05.688835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:36:05.688835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:05.692480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 207/422] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:05.694581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:05.698545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50152b-f8c1-4d36-84fc-528b0ac4dff9 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:05.802390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 216/423] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 03:36:05.809146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:05.809601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809353eb-5758-4be3-9264-f79e51e482af None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:05.821273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50152b-f8c1-4d36-84fc-528b0ac4dff9 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:05.866314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00500fa6-7c61-4415-b643-ace795f8519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:05.867486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00500fa6-7c61-4415-b643-ace795f8519c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:05.868177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00500fa6-7c61-4415-b643-ace795f8519c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:05.868557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00500fa6-7c61-4415-b643-ace795f8519c None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:05.869320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00500fa6-7c61-4415-b643-ace795f8519c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:05.869627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00500fa6-7c61-4415-b643-ace795f8519c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:05.871248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00500fa6-7c61-4415-b643-ace795f8519c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:05.885534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 208/424] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:05.887102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00500fa6-7c61-4415-b643-ace795f8519c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:06.003734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809353eb-5758-4be3-9264-f79e51e482af None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:06.011571 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:06.060629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82120dea-31d4-42c2-8fa2-27858f3a9588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:06.061560 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82120dea-31d4-42c2-8fa2-27858f3a9588 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:06.061659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00500fa6-7c61-4415-b643-ace795f8519c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:06.061835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82120dea-31d4-42c2-8fa2-27858f3a9588 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:06.062109 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82120dea-31d4-42c2-8fa2-27858f3a9588 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:06.063528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82120dea-31d4-42c2-8fa2-27858f3a9588 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:06.065309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82120dea-31d4-42c2-8fa2-27858f3a9588 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:06.067506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 217/425] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:36:05 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:06.072660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82120dea-31d4-42c2-8fa2-27858f3a9588 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:06.101036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424551e-43bc-4e50-956d-6be1a50d4a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:06.101793 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424551e-43bc-4e50-956d-6be1a50d4a7c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:06.102084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424551e-43bc-4e50-956d-6be1a50d4a7c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:06.102447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424551e-43bc-4e50-956d-6be1a50d4a7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:06.122214 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:06.172848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424551e-43bc-4e50-956d-6be1a50d4a7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:06.174436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424551e-43bc-4e50-956d-6be1a50d4a7c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:06.278867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 209/426] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 03:36:06.288949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424551e-43bc-4e50-956d-6be1a50d4a7c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:06.301456 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82120dea-31d4-42c2-8fa2-27858f3a9588 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:06.360388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2036b-55a6-4c2a-b235-a49ce9eb8b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:06.361520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2036b-55a6-4c2a-b235-a49ce9eb8b38 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:06.361935 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2036b-55a6-4c2a-b235-a49ce9eb8b38 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:06.362339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf2036b-55a6-4c2a-b235-a49ce9eb8b38 None admin] PATH_INFO: `/v3/groups/35044f9f91b24a7c98b9568355201b66` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:06.363776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf2036b-55a6-4c2a-b235-a49ce9eb8b38 None admin] RBAC: Authorizing `identity:get_group(group_id=35044f9f91b24a7c98b9568355201b66)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:06.365391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf2036b-55a6-4c2a-b235-a49ce9eb8b38 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:06.369676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 218/427] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:36:06 2022] GET /identity/v3/groups/35044f9f91b24a7c98b9568355201b66 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:06.374134 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2036b-55a6-4c2a-b235-a49ce9eb8b38 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:06.384373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:06.443796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424551e-43bc-4e50-956d-6be1a50d4a7c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:06.460033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:06.479773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10466b9-ee78-4ccb-9d32-ac70217439e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:06.480958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10466b9-ee78-4ccb-9d32-ac70217439e5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:06.481366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10466b9-ee78-4ccb-9d32-ac70217439e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:06.481737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10466b9-ee78-4ccb-9d32-ac70217439e5 None admin] PATH_INFO: `/v3/projects/b81760932eec4beda4343070aaf9b956` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:06.483170 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10466b9-ee78-4ccb-9d32-ac70217439e5 None admin] RBAC: Authorizing `identity:get_project(project_id=b81760932eec4beda4343070aaf9b956)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:06.484795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10466b9-ee78-4ccb-9d32-ac70217439e5 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:06.490970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 210/428] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:36:06 2022] GET /identity/v3/projects/b81760932eec4beda4343070aaf9b956 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:06.493147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10466b9-ee78-4ccb-9d32-ac70217439e5 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:06.583716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf2036b-55a6-4c2a-b235-a49ce9eb8b38 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:06.656764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23caad4-eca3-4f91-a7c8-0b8db864e470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:06.657435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23caad4-eca3-4f91-a7c8-0b8db864e470 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:06.657713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23caad4-eca3-4f91-a7c8-0b8db864e470 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:06.657983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23caad4-eca3-4f91-a7c8-0b8db864e470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:06.658836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23caad4-eca3-4f91-a7c8-0b8db864e470 None admin] RBAC: Adding query filter params (group.id=35044f9f91b24a7c98b9568355201b66, role.id=8f3ea95f038d4d8fbd1fd4897be36344, scope.project.id=b81760932eec4beda4343070aaf9b956) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:06.659146 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23caad4-eca3-4f91-a7c8-0b8db864e470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:06.660636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23caad4-eca3-4f91-a7c8-0b8db864e470 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:06.674424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 219/429] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 12 03:36:06 2022] GET /identity/v3/role_assignments?group.id=35044f9f91b24a7c98b9568355201b66&scope.project.id=b81760932eec4beda4343070aaf9b956&role.id=8f3ea95f038d4d8fbd1fd4897be36344 => generated 611 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:06.677129 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10466b9-ee78-4ccb-9d32-ac70217439e5 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:06.716066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:06.751712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95196188-3d96-454d-98a7-ef253ca84812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:06.752629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95196188-3d96-454d-98a7-ef253ca84812 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:06.752914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95196188-3d96-454d-98a7-ef253ca84812 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:06.753190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95196188-3d96-454d-98a7-ef253ca84812 None admin] PATH_INFO: `/v3/groups/35044f9f91b24a7c98b9568355201b66` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:06.756803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95196188-3d96-454d-98a7-ef253ca84812 None admin] RBAC: Authorizing `identity:get_group(group_id=35044f9f91b24a7c98b9568355201b66)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:06.758371 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95196188-3d96-454d-98a7-ef253ca84812 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:06.760302 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 211/430] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:36:06 2022] GET /identity/v3/groups/35044f9f91b24a7c98b9568355201b66 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:06.766624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23caad4-eca3-4f91-a7c8-0b8db864e470 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:06.887111 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23caad4-eca3-4f91-a7c8-0b8db864e470 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:06.889441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:06.923790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf73829-a7eb-4ab4-9b31-bcdcd7f8a90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:06.924486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf73829-a7eb-4ab4-9b31-bcdcd7f8a90d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:06.924768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf73829-a7eb-4ab4-9b31-bcdcd7f8a90d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:06.925046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf73829-a7eb-4ab4-9b31-bcdcd7f8a90d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:06.925912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf73829-a7eb-4ab4-9b31-bcdcd7f8a90d None admin] RBAC: Adding query filter params (group.id=35044f9f91b24a7c98b9568355201b66, role.id=ff6066b1fb8940cd9ec82c078fd8d2f6, scope.project.id=b81760932eec4beda4343070aaf9b956) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:06.926232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf73829-a7eb-4ab4-9b31-bcdcd7f8a90d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:06.927745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf73829-a7eb-4ab4-9b31-bcdcd7f8a90d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:06.931200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877ff72-467f-46ec-b5c6-5d40c022b74a None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:06.931490 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877ff72-467f-46ec-b5c6-5d40c022b74a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:06.931769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877ff72-467f-46ec-b5c6-5d40c022b74a None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:06.933546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 212/431] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:06.937836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 220/432] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 12 03:36:06 2022] GET /identity/v3/role_assignments?group.id=35044f9f91b24a7c98b9568355201b66&scope.project.id=b81760932eec4beda4343070aaf9b956&role.id=ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:06.941227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec21f4-d4e7-4a1f-bb9a-525711153b68 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:06.941227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec21f4-d4e7-4a1f-bb9a-525711153b68 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:06.941227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec21f4-d4e7-4a1f-bb9a-525711153b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:06.973672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-adec21f4-d4e7-4a1f-bb9a-525711153b68 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:06.974889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-adec21f4-d4e7-4a1f-bb9a-525711153b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:07.062902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 213/433] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:07.086005 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf73829-a7eb-4ab4-9b31-bcdcd7f8a90d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:07.199507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf73829-a7eb-4ab4-9b31-bcdcd7f8a90d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:07.239238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc86ad-c314-42fa-8af4-b2b2e31f188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:07.240638 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc86ad-c314-42fa-8af4-b2b2e31f188d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:07.240638 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc86ad-c314-42fa-8af4-b2b2e31f188d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:07.240638 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc86ad-c314-42fa-8af4-b2b2e31f188d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:07.252138 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc86ad-c314-42fa-8af4-b2b2e31f188d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-72dc86ad-c314-42fa-8af4-b2b2e31f188d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:07.260061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:07.260988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:07.261787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 221/434] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:07.267464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec21f4-d4e7-4a1f-bb9a-525711153b68 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:07.374789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec21f4-d4e7-4a1f-bb9a-525711153b68 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:07.396891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:07.420700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fc226-76fd-40a9-97cd-641150555d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:07.421393 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fc226-76fd-40a9-97cd-641150555d2f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:07.421679 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fc226-76fd-40a9-97cd-641150555d2f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:07.421959 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fc226-76fd-40a9-97cd-641150555d2f None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:07.422749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fc226-76fd-40a9-97cd-641150555d2f None admin] RBAC: Adding query filter params (name=member) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:07.423058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fc226-76fd-40a9-97cd-641150555d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:07.424512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fc226-76fd-40a9-97cd-641150555d2f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:07.433991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 214/435] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:36:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:07.440207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc86ad-c314-42fa-8af4-b2b2e31f188d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:07.474296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:07.556445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc86ad-c314-42fa-8af4-b2b2e31f188d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:07.593797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ef9f4-fac9-4537-a8ce-0bbdab999aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:07.594568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ef9f4-fac9-4537-a8ce-0bbdab999aa8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:07.594730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ef9f4-fac9-4537-a8ce-0bbdab999aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:07.595008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ef9f4-fac9-4537-a8ce-0bbdab999aa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:07.666591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ef9f4-fac9-4537-a8ce-0bbdab999aa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:07.668105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ef9f4-fac9-4537-a8ce-0bbdab999aa8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:07.695959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:07.712334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ef3a6-0a08-40e0-863a-2f3c8d9851fd None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:07.712615 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ef3a6-0a08-40e0-863a-2f3c8d9851fd None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:07.712884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ef3a6-0a08-40e0-863a-2f3c8d9851fd None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:07.714256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 215/436] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:07.719006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c07a8-fa1d-4f05-905e-b69007bd1931 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:07.719294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c07a8-fa1d-4f05-905e-b69007bd1931 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:07.719574 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3c07a8-fa1d-4f05-905e-b69007bd1931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:07.737134 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-eb3c07a8-fa1d-4f05-905e-b69007bd1931 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:07.738223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-eb3c07a8-fa1d-4f05-905e-b69007bd1931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:07.776093 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 222/437] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:07.781469 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ef9f4-fac9-4537-a8ce-0bbdab999aa8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:07.817755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 216/438] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:36:07.840572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c07a8-fa1d-4f05-905e-b69007bd1931 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:07.896785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ef9f4-fac9-4537-a8ce-0bbdab999aa8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:07.926974 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:07.934741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12228371-3b46-4716-9081-41b80b378cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:07.935600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12228371-3b46-4716-9081-41b80b378cac None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:07.935876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12228371-3b46-4716-9081-41b80b378cac None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:07.936180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12228371-3b46-4716-9081-41b80b378cac None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:07.937343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12228371-3b46-4716-9081-41b80b378cac None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:07.938807 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12228371-3b46-4716-9081-41b80b378cac None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:07.940510 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 223/439] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:36:07 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:07.943442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3c07a8-fa1d-4f05-905e-b69007bd1931 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:07.944949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375ea73-ac6e-4395-9111-c64ca77dde0d None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:07.945228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375ea73-ac6e-4395-9111-c64ca77dde0d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:07.945600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375ea73-ac6e-4395-9111-c64ca77dde0d None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:07.946886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 224/440] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:07.947573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375ea73-ac6e-4395-9111-c64ca77dde0d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:07.953386 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:07.978776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e85b85-de5d-4c7a-9fe4-9994d2a01197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:07.979609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e85b85-de5d-4c7a-9fe4-9994d2a01197 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:07.979898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e85b85-de5d-4c7a-9fe4-9994d2a01197 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:07.980207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e85b85-de5d-4c7a-9fe4-9994d2a01197 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:07.990125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e85b85-de5d-4c7a-9fe4-9994d2a01197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-52e85b85-de5d-4c7a-9fe4-9994d2a01197 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:07.997690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:07.998765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 217/441] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:08.004563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e1aee-500e-4ff8-9d6f-2f17f52dc0d3 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:08.005260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e1aee-500e-4ff8-9d6f-2f17f52dc0d3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:08.005260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e1aee-500e-4ff8-9d6f-2f17f52dc0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:08.016674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2f0e1aee-500e-4ff8-9d6f-2f17f52dc0d3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:08.017838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2f0e1aee-500e-4ff8-9d6f-2f17f52dc0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:08.049686 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375ea73-ac6e-4395-9111-c64ca77dde0d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:08.084158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 218/442] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:08.084966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e1aee-500e-4ff8-9d6f-2f17f52dc0d3 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:08.085090 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e91758-d2c0-40c1-894e-d21070864876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:08.085874 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e91758-d2c0-40c1-894e-d21070864876 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:08.086249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e91758-d2c0-40c1-894e-d21070864876 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:08.086597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e91758-d2c0-40c1-894e-d21070864876 None admin] PATH_INFO: `/v3/groups/35044f9f91b24a7c98b9568355201b66` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:08.087876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e91758-d2c0-40c1-894e-d21070864876 None admin] RBAC: Authorizing `identity:get_group(group_id=35044f9f91b24a7c98b9568355201b66)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:08.089362 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e91758-d2c0-40c1-894e-d21070864876 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:08.091012 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 225/443] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:36:07 2022] GET /identity/v3/groups/35044f9f91b24a7c98b9568355201b66 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:08.096918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e91758-d2c0-40c1-894e-d21070864876 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:08.166707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:08.184137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:08.188150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e1aee-500e-4ff8-9d6f-2f17f52dc0d3 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:08.206461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e91758-d2c0-40c1-894e-d21070864876 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:08.221534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b61c45-39b4-453b-9797-05f07ae21c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:08.222205 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b61c45-39b4-453b-9797-05f07ae21c6c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:08.222495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b61c45-39b4-453b-9797-05f07ae21c6c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:08.222766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b61c45-39b4-453b-9797-05f07ae21c6c None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:08.223520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b61c45-39b4-453b-9797-05f07ae21c6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:08.223817 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b61c45-39b4-453b-9797-05f07ae21c6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:08.225267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b61c45-39b4-453b-9797-05f07ae21c6c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:08.233723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 219/444] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:08.234485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b61c45-39b4-453b-9797-05f07ae21c6c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:08.244182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058b988-74d2-406e-a209-b74fd373357d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:08.244799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b988-74d2-406e-a209-b74fd373357d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:08.245066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b988-74d2-406e-a209-b74fd373357d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:08.245334 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058b988-74d2-406e-a209-b74fd373357d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:08.246169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058b988-74d2-406e-a209-b74fd373357d None admin] RBAC: Adding query filter params (group.id=35044f9f91b24a7c98b9568355201b66, role.id=8f3ea95f038d4d8fbd1fd4897be36344, scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:08.246476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058b988-74d2-406e-a209-b74fd373357d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:08.247903 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058b988-74d2-406e-a209-b74fd373357d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:08.253946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 226/445] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 12 03:36:08 2022] GET /identity/v3/role_assignments?group.id=35044f9f91b24a7c98b9568355201b66&scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62&role.id=8f3ea95f038d4d8fbd1fd4897be36344 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:08.254759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058b988-74d2-406e-a209-b74fd373357d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:08.352728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b61c45-39b4-453b-9797-05f07ae21c6c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:08.371472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058b988-74d2-406e-a209-b74fd373357d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:08.389722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34608e66-3080-4ebc-923d-fd67c591b984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:08.390544 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34608e66-3080-4ebc-923d-fd67c591b984 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:08.390821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34608e66-3080-4ebc-923d-fd67c591b984 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:08.391151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34608e66-3080-4ebc-923d-fd67c591b984 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:08.400189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:08.405196 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34608e66-3080-4ebc-923d-fd67c591b984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:08.409656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6043cc-eb3f-422d-bf6e-b6a9ab44d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:08.410316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6043cc-eb3f-422d-bf6e-b6a9ab44d6df None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:08.410600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6043cc-eb3f-422d-bf6e-b6a9ab44d6df None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:08.410874 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6043cc-eb3f-422d-bf6e-b6a9ab44d6df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-34608e66-3080-4ebc-923d-fd67c591b984 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:08.413242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:08.414198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:08.414964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 220/446] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:08.420529 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34608e66-3080-4ebc-923d-fd67c591b984 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:08.447445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:08.494728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6043cc-eb3f-422d-bf6e-b6a9ab44d6df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:08.496165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6043cc-eb3f-422d-bf6e-b6a9ab44d6df None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:08.537811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34608e66-3080-4ebc-923d-fd67c591b984 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:08.578055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e941bc0-b72f-4a01-b096-8e0bf9637fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:08.578288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e941bc0-b72f-4a01-b096-8e0bf9637fed None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:08.579584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e941bc0-b72f-4a01-b096-8e0bf9637fed None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:08.579584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e941bc0-b72f-4a01-b096-8e0bf9637fed None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:08.579995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e941bc0-b72f-4a01-b096-8e0bf9637fed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:08.581828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e941bc0-b72f-4a01-b096-8e0bf9637fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:08.582164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e941bc0-b72f-4a01-b096-8e0bf9637fed None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:08.592327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 221/447] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:08.597883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e941bc0-b72f-4a01-b096-8e0bf9637fed None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:08.610263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 227/448] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:08.616370 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6043cc-eb3f-422d-bf6e-b6a9ab44d6df None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:08.644669 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:08.682082 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:08.700786 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e941bc0-b72f-4a01-b096-8e0bf9637fed None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:08.714330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6043cc-eb3f-422d-bf6e-b6a9ab44d6df None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:08.733613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e809cd-c825-4059-9659-29a120767c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:08.734247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e809cd-c825-4059-9659-29a120767c76 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:08.734532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e809cd-c825-4059-9659-29a120767c76 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:08.734803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e809cd-c825-4059-9659-29a120767c76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:08.748611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05534633-5c6c-4984-9fae-3d0f227a2c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:08.749401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05534633-5c6c-4984-9fae-3d0f227a2c4f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:08.749682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05534633-5c6c-4984-9fae-3d0f227a2c4f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:08.749950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05534633-5c6c-4984-9fae-3d0f227a2c4f None admin] PATH_INFO: `/v3/projects/579b7a08aae94d3fa59617c603862f30` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:08.751197 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05534633-5c6c-4984-9fae-3d0f227a2c4f None admin] RBAC: Authorizing `identity:get_project(project_id=579b7a08aae94d3fa59617c603862f30)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:08.752757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05534633-5c6c-4984-9fae-3d0f227a2c4f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:08.754813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 228/449] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:36:08 2022] GET /identity/v3/projects/579b7a08aae94d3fa59617c603862f30 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:08.759867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05534633-5c6c-4984-9fae-3d0f227a2c4f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:08.804519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e809cd-c825-4059-9659-29a120767c76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:08.805994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e809cd-c825-4059-9659-29a120767c76 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:08.850885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:08.857307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05534633-5c6c-4984-9fae-3d0f227a2c4f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:08.883872 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:08.891550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae57d8-7601-4703-a7dc-8dd10424427e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:08.892436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae57d8-7601-4703-a7dc-8dd10424427e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:08.892719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae57d8-7601-4703-a7dc-8dd10424427e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:08.892993 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae57d8-7601-4703-a7dc-8dd10424427e None admin] PATH_INFO: `/v3/groups/3d767ccc710b4546a820a8c65becf34b` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:08.894509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae57d8-7601-4703-a7dc-8dd10424427e None admin] RBAC: Authorizing `identity:get_group(group_id=3d767ccc710b4546a820a8c65becf34b)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:08.895965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae57d8-7601-4703-a7dc-8dd10424427e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:08.897729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 229/450] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:36:08 2022] GET /identity/v3/groups/3d767ccc710b4546a820a8c65becf34b => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:08.903444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae57d8-7601-4703-a7dc-8dd10424427e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:08.903570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 222/451] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 03:36:08.908777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e809cd-c825-4059-9659-29a120767c76 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:09.002639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae57d8-7601-4703-a7dc-8dd10424427e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:09.007843 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e809cd-c825-4059-9659-29a120767c76 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:09.035737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372504ee-8a53-42ab-9bca-2f654584b3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:09.036417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372504ee-8a53-42ab-9bca-2f654584b3a9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:09.036702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372504ee-8a53-42ab-9bca-2f654584b3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:09.036975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372504ee-8a53-42ab-9bca-2f654584b3a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:09.037785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372504ee-8a53-42ab-9bca-2f654584b3a9 None admin] RBAC: Adding query filter params (group.id=3d767ccc710b4546a820a8c65becf34b, role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=579b7a08aae94d3fa59617c603862f30) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:09.038088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372504ee-8a53-42ab-9bca-2f654584b3a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:09.039637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372504ee-8a53-42ab-9bca-2f654584b3a9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:09.040597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70470b-4394-4829-a79d-2f84b1881ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:09.041459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70470b-4394-4829-a79d-2f84b1881ea5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:09.041743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70470b-4394-4829-a79d-2f84b1881ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:09.042015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70470b-4394-4829-a79d-2f84b1881ea5 None admin] PATH_INFO: `/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:09.043225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70470b-4394-4829-a79d-2f84b1881ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=3bcbe8d7aaa3411b844ce5c42db14e62)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:09.044691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70470b-4394-4829-a79d-2f84b1881ea5 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:09.045431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 230/452] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 12 03:36:08 2022] GET /identity/v3/role_assignments?group.id=3d767ccc710b4546a820a8c65becf34b&scope.project.id=579b7a08aae94d3fa59617c603862f30&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:09.050862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 223/453] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:36:08 2022] GET /identity/v3/projects/3bcbe8d7aaa3411b844ce5c42db14e62 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:09.052147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372504ee-8a53-42ab-9bca-2f654584b3a9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:09.083451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:09.167176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372504ee-8a53-42ab-9bca-2f654584b3a9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:09.204371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a6f4f-24a0-4d2a-b9b8-22ca0fd664ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:09.205211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a6f4f-24a0-4d2a-b9b8-22ca0fd664ec None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:09.205495 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a6f4f-24a0-4d2a-b9b8-22ca0fd664ec None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:09.205772 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a6f4f-24a0-4d2a-b9b8-22ca0fd664ec None admin] PATH_INFO: `/v3/groups/35044f9f91b24a7c98b9568355201b66` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:09.206958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a6f4f-24a0-4d2a-b9b8-22ca0fd664ec None admin] RBAC: Authorizing `identity:get_group(group_id=35044f9f91b24a7c98b9568355201b66)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:09.208459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a6f4f-24a0-4d2a-b9b8-22ca0fd664ec None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:09.210459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 231/454] 10.222.0.20 () {56 vars in 1180 bytes} [Tue Jul 12 03:36:09 2022] GET /identity/v3/groups/35044f9f91b24a7c98b9568355201b66 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:09.216211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70470b-4394-4829-a79d-2f84b1881ea5 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:09.267524 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:09.317016 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70470b-4394-4829-a79d-2f84b1881ea5 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:09.352784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c40aa3-2fbf-45ef-b54b-1d3628acd863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:09.353817 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40aa3-2fbf-45ef-b54b-1d3628acd863 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:09.354268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40aa3-2fbf-45ef-b54b-1d3628acd863 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:09.354582 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40aa3-2fbf-45ef-b54b-1d3628acd863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:09.355491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40aa3-2fbf-45ef-b54b-1d3628acd863 None admin] RBAC: Adding query filter params (group.id=35044f9f91b24a7c98b9568355201b66, role.id=ff6066b1fb8940cd9ec82c078fd8d2f6, scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:09.355802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40aa3-2fbf-45ef-b54b-1d3628acd863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:09.357559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40aa3-2fbf-45ef-b54b-1d3628acd863 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:09.364350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 224/455] 10.222.0.20 () {56 vars in 1376 bytes} [Tue Jul 12 03:36:09 2022] GET /identity/v3/role_assignments?group.id=35044f9f91b24a7c98b9568355201b66&scope.project.id=3bcbe8d7aaa3411b844ce5c42db14e62&role.id=ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:11.897142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74425030-16e7-41b8-8aa0-0fc5f44a43c6 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:11.897783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74425030-16e7-41b8-8aa0-0fc5f44a43c6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:11.897783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74425030-16e7-41b8-8aa0-0fc5f44a43c6 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:11.899422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 232/456] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:11.904569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb29493-bfb7-42f1-98ce-2d7a3de4e63a None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:11.905389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb29493-bfb7-42f1-98ce-2d7a3de4e63a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:11.905861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb29493-bfb7-42f1-98ce-2d7a3de4e63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:11.929095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-9fb29493-bfb7-42f1-98ce-2d7a3de4e63a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:11.930609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-9fb29493-bfb7-42f1-98ce-2d7a3de4e63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:12.027640 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 225/457] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:12.063618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74425030-16e7-41b8-8aa0-0fc5f44a43c6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:12.107024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78bf1f-f0b9-45af-910c-609ad3010537 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:12.107351 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78bf1f-f0b9-45af-910c-609ad3010537 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:12.107570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78bf1f-f0b9-45af-910c-609ad3010537 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:12.109329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 226/458] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:12.113919 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fbb14-aced-4b9b-a2af-f0f5e691bc95 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:12.114308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fbb14-aced-4b9b-a2af-f0f5e691bc95 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:12.114697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fbb14-aced-4b9b-a2af-f0f5e691bc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:12.144913 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-c57fbb14-aced-4b9b-a2af-f0f5e691bc95 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:12.146957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c57fbb14-aced-4b9b-a2af-f0f5e691bc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:12.222503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:12.237223 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:12.239831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74425030-16e7-41b8-8aa0-0fc5f44a43c6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:12.278667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6ce67-0b3a-4f9e-952a-54cd66bffeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:12.279569 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6ce67-0b3a-4f9e-952a-54cd66bffeb6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:12.279916 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6ce67-0b3a-4f9e-952a-54cd66bffeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:12.280283 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6ce67-0b3a-4f9e-952a-54cd66bffeb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:12.289689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 227/459] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:36:12.291154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6ce67-0b3a-4f9e-952a-54cd66bffeb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:12.292743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6ce67-0b3a-4f9e-952a-54cd66bffeb6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f7f6ce67-0b3a-4f9e-952a-54cd66bffeb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:12.311547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:12.312452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:12.313270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:12.313270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:12.313270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:12.313270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:12.313270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 233/460] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:12.319772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57fbb14-aced-4b9b-a2af-f0f5e691bc95 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:12.326654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6ce67-0b3a-4f9e-952a-54cd66bffeb6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:12.457162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6ce67-0b3a-4f9e-952a-54cd66bffeb6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:12.501043 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68b649-9766-4764-b33a-e58c686e078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:12.502271 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b649-9766-4764-b33a-e58c686e078a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:12.502554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b649-9766-4764-b33a-e58c686e078a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:12.503089 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b649-9766-4764-b33a-e58c686e078a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:12.515895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68b649-9766-4764-b33a-e58c686e078a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:12.517411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68b649-9766-4764-b33a-e58c686e078a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:12.527902 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-5f68b649-9766-4764-b33a-e58c686e078a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:12.536714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:12.537799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:12.538836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:12.538836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:12.538836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:12.538836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:12.540734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 234/461] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:12.545320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c070fec-bbb3-4f73-ad60-31ee6cf72038 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:12.545320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c070fec-bbb3-4f73-ad60-31ee6cf72038 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:12.545320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c070fec-bbb3-4f73-ad60-31ee6cf72038 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:12.547749 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 235/462] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:12.548668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c070fec-bbb3-4f73-ad60-31ee6cf72038 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:12.549255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57fbb14-aced-4b9b-a2af-f0f5e691bc95 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:12.587824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e460e-0e0a-43c0-8ab0-a6a602e20821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:12.588864 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e460e-0e0a-43c0-8ab0-a6a602e20821 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:12.589148 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e460e-0e0a-43c0-8ab0-a6a602e20821 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:12.589423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e460e-0e0a-43c0-8ab0-a6a602e20821 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:12.590147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e460e-0e0a-43c0-8ab0-a6a602e20821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:12.590452 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e460e-0e0a-43c0-8ab0-a6a602e20821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:12.591853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e460e-0e0a-43c0-8ab0-a6a602e20821 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:12.608606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 228/463] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:12.608606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090981fd-7136-415a-a9f9-ca497e092811 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:12.608606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090981fd-7136-415a-a9f9-ca497e092811 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:12.608606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090981fd-7136-415a-a9f9-ca497e092811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:12.624313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-090981fd-7136-415a-a9f9-ca497e092811 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:12.625487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-090981fd-7136-415a-a9f9-ca497e092811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:12.654327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:12.657709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c070fec-bbb3-4f73-ad60-31ee6cf72038 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:12.692700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2899c-5426-4730-80b9-1639388db2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:12.693384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2899c-5426-4730-80b9-1639388db2c5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:12.693664 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2899c-5426-4730-80b9-1639388db2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:12.694027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2899c-5426-4730-80b9-1639388db2c5 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:12.694770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2899c-5426-4730-80b9-1639388db2c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:12.695071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2899c-5426-4730-80b9-1639388db2c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:12.700285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 229/464] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:12.700451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2899c-5426-4730-80b9-1639388db2c5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:12.700534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24be1b0-7d83-40d1-9f89-d0a1b76e2d5f None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:12.700534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24be1b0-7d83-40d1-9f89-d0a1b76e2d5f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:12.700534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24be1b0-7d83-40d1-9f89-d0a1b76e2d5f None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:12.700534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 230/465] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:12.700534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24be1b0-7d83-40d1-9f89-d0a1b76e2d5f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:12.714627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 236/466] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:12.716059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96496ebf-8c70-42b6-94d2-aca2332fef96 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:12.716351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96496ebf-8c70-42b6-94d2-aca2332fef96 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:12.716627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96496ebf-8c70-42b6-94d2-aca2332fef96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:12.734391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-96496ebf-8c70-42b6-94d2-aca2332fef96 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:12.735581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-96496ebf-8c70-42b6-94d2-aca2332fef96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:12.794545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:12.807302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 237/467] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:12.808349 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96496ebf-8c70-42b6-94d2-aca2332fef96 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:12.817030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24be1b0-7d83-40d1-9f89-d0a1b76e2d5f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:12.853356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7d6e3-8123-4215-8214-0ee85dcdee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:12.854058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7d6e3-8123-4215-8214-0ee85dcdee55 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:12.854336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7d6e3-8123-4215-8214-0ee85dcdee55 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:12.854609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7d6e3-8123-4215-8214-0ee85dcdee55 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:12.855299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7d6e3-8123-4215-8214-0ee85dcdee55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:12.856780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7d6e3-8123-4215-8214-0ee85dcdee55 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:12.888027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 231/468] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:36:12 2022] POST /identity/v3/users => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:36:12.888027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7d6e3-8123-4215-8214-0ee85dcdee55 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:12.912013 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:12.959881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96496ebf-8c70-42b6-94d2-aca2332fef96 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:13.010113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80400393-25f1-4f35-a8b7-39b7866fcf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:13.010819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80400393-25f1-4f35-a8b7-39b7866fcf10 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:13.011105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80400393-25f1-4f35-a8b7-39b7866fcf10 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:13.011381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80400393-25f1-4f35-a8b7-39b7866fcf10 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:13.012426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80400393-25f1-4f35-a8b7-39b7866fcf10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:13.014223 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80400393-25f1-4f35-a8b7-39b7866fcf10 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:13.022055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7d6e3-8123-4215-8214-0ee85dcdee55 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:13.053921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 238/469] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:36:12 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:36:13.054932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80400393-25f1-4f35-a8b7-39b7866fcf10 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:13.082010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abca9b-c273-4b22-ac66-e24b5cb6c8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:13.083284 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abca9b-c273-4b22-ac66-e24b5cb6c8f4 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:13.083575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abca9b-c273-4b22-ac66-e24b5cb6c8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:13.083854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abca9b-c273-4b22-ac66-e24b5cb6c8f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:13.099230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abca9b-c273-4b22-ac66-e24b5cb6c8f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:13.100860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abca9b-c273-4b22-ac66-e24b5cb6c8f4 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:13.119408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-e5abca9b-c273-4b22-ac66-e24b5cb6c8f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:13.124267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:13.125172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:13.126506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:13.126506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:13.126506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:13.126506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:13.126506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 232/470] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:13.133394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abca9b-c273-4b22-ac66-e24b5cb6c8f4 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:13.190151 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80400393-25f1-4f35-a8b7-39b7866fcf10 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:13.217823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:13.236667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15513026-1525-4908-b811-fa319ec9b0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:13.237675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15513026-1525-4908-b811-fa319ec9b0f0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:13.237957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15513026-1525-4908-b811-fa319ec9b0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:13.238227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15513026-1525-4908-b811-fa319ec9b0f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:13.251018 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15513026-1525-4908-b811-fa319ec9b0f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:13.252461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15513026-1525-4908-b811-fa319ec9b0f0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:13.257829 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abca9b-c273-4b22-ac66-e24b5cb6c8f4 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-15513026-1525-4908-b811-fa319ec9b0f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:13.271247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:13.272272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:13.273528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:13.273528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:13.273528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:13.273528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:13.273528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 239/471] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:13.278378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15513026-1525-4908-b811-fa319ec9b0f0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:13.301352 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929122d-845e-44c0-a2e6-9e0e7117611c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:13.302455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929122d-845e-44c0-a2e6-9e0e7117611c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:13.302879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929122d-845e-44c0-a2e6-9e0e7117611c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:13.303358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929122d-845e-44c0-a2e6-9e0e7117611c None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:13.304312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929122d-845e-44c0-a2e6-9e0e7117611c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:13.304778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929122d-845e-44c0-a2e6-9e0e7117611c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:13.306518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929122d-845e-44c0-a2e6-9e0e7117611c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:13.321184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 233/472] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:13.333540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929122d-845e-44c0-a2e6-9e0e7117611c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:13.386327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:13.390527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15513026-1525-4908-b811-fa319ec9b0f0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:13.426887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058c134-2782-4042-87cd-b32a0acf5ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:13.427559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058c134-2782-4042-87cd-b32a0acf5ab0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:13.427886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058c134-2782-4042-87cd-b32a0acf5ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:13.428255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058c134-2782-4042-87cd-b32a0acf5ab0 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:13.429065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058c134-2782-4042-87cd-b32a0acf5ab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:13.429371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058c134-2782-4042-87cd-b32a0acf5ab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:13.430758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058c134-2782-4042-87cd-b32a0acf5ab0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:13.447782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 240/473] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:13.452967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058c134-2782-4042-87cd-b32a0acf5ab0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:13.487023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929122d-845e-44c0-a2e6-9e0e7117611c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:13.524285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:13.535514 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80203830-a2be-439e-8105-f01db5f2321b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:13.536238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80203830-a2be-439e-8105-f01db5f2321b None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:13.536520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80203830-a2be-439e-8105-f01db5f2321b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:13.536795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80203830-a2be-439e-8105-f01db5f2321b None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:13.537697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80203830-a2be-439e-8105-f01db5f2321b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:13.539157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80203830-a2be-439e-8105-f01db5f2321b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:13.577568 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 234/474] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:36:13 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:36:13.578881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2975332-2915-40f1-974d-31c1df7bc5c3 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:13.579316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2975332-2915-40f1-974d-31c1df7bc5c3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:13.579643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2975332-2915-40f1-974d-31c1df7bc5c3 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:13.581649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 235/475] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:13.600703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709424e-3587-4507-9716-078eaf0314fa None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:13.601013 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709424e-3587-4507-9716-078eaf0314fa None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:13.601291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709424e-3587-4507-9716-078eaf0314fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:13.634482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-7709424e-3587-4507-9716-078eaf0314fa None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:13.635724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7709424e-3587-4507-9716-078eaf0314fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:13.647659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058c134-2782-4042-87cd-b32a0acf5ab0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:13.709915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d43986-12a0-4961-9b28-99044091b1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:13.710613 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d43986-12a0-4961-9b28-99044091b1fa None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:13.710895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d43986-12a0-4961-9b28-99044091b1fa None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:13.711183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d43986-12a0-4961-9b28-99044091b1fa None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:13.711884 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d43986-12a0-4961-9b28-99044091b1fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:13.713385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d43986-12a0-4961-9b28-99044091b1fa None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:13.735427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 236/476] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:36:13.758816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 241/477] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:36:13 2022] POST /identity/v3/users => generated 249 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:36:13.773850 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709424e-3587-4507-9716-078eaf0314fa None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:13.887960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709424e-3587-4507-9716-078eaf0314fa None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:13.912525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:13.933453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56845914-8441-457b-a338-b311d0154619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:13.934147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56845914-8441-457b-a338-b311d0154619 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:13.934434 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56845914-8441-457b-a338-b311d0154619 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:13.934709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56845914-8441-457b-a338-b311d0154619 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:13.935575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56845914-8441-457b-a338-b311d0154619 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:13.937391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56845914-8441-457b-a338-b311d0154619 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:13.947099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.sql.core [None req-56845914-8441-457b-a338-b311d0154619 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 03:36:13.947099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 03:36:13.947099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [parameters: {'id': '09517617678d49e6ac5f6b0bdb5c9766', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 03:36:13.947099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86935) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-56845914-8441-457b-a338-b311d0154619 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application conn.query(q) Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result.read() Jul 12 03:36:13.964063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return ref.to_dict() Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 03:36:13.964889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application next(self.gen) Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application yield resource Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application next(self.gen) Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise self.value Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application session.commit() Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 03:36:13.965787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.session.flush() Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application compat.raise_( Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 03:36:13.966636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 03:36:13.967500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application conn.query(q) Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result.read() Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 03:36:13.969625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [parameters: {'id': '09517617678d49e6ac5f6b0bdb5c9766', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:13.971785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 03:36:13.973860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:13.977139 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 237/478] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Jul 12 03:36:13 2022] POST /identity/v3/roles => generated 144 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:13.977294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d43986-12a0-4961-9b28-99044091b1fa None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:13.991129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:14.089098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d43986-12a0-4961-9b28-99044091b1fa None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:14.125599 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b14f6-fb8b-4afd-9209-6028704e9623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:14.126778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b14f6-fb8b-4afd-9209-6028704e9623 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:14.127257 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b14f6-fb8b-4afd-9209-6028704e9623 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:14.127818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b14f6-fb8b-4afd-9209-6028704e9623 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:14.139002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b14f6-fb8b-4afd-9209-6028704e9623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c13b14f6-fb8b-4afd-9209-6028704e9623 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:14.147838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:14.148801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:14.149621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:14.149621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:14.149621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:14.149621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:14.149621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:14.149621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:14.149621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:14.149621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:14.149621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:14.149621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:14.149621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 03:36:14.149621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:14.152356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 242/479] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:14.157767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56845914-8441-457b-a338-b311d0154619 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:14.309155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56845914-8441-457b-a338-b311d0154619 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:14.360732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f6cbef-6fe6-496f-a049-b376b972e3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:14.361440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6cbef-6fe6-496f-a049-b376b972e3ae None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:14.361727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6cbef-6fe6-496f-a049-b376b972e3ae None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:14.362002 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6cbef-6fe6-496f-a049-b376b972e3ae None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:14.362777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6cbef-6fe6-496f-a049-b376b972e3ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:14.363098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6cbef-6fe6-496f-a049-b376b972e3ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:14.364543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f6cbef-6fe6-496f-a049-b376b972e3ae None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:14.374428 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 238/480] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:15.404833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532ea4d-5870-4e25-b229-323ccf55c957 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:15.405877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532ea4d-5870-4e25-b229-323ccf55c957 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:15.406385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532ea4d-5870-4e25-b229-323ccf55c957 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:15.408245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 243/481] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:15.413217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40066a45-747f-4a7b-8111-03b802afa8e2 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:15.414034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40066a45-747f-4a7b-8111-03b802afa8e2 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:15.414507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40066a45-747f-4a7b-8111-03b802afa8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:15.437085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-40066a45-747f-4a7b-8111-03b802afa8e2 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:15.438613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-40066a45-747f-4a7b-8111-03b802afa8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:15.485378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af06e25-fc06-4efb-bf4a-b0559b38b1ed None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:15.485716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af06e25-fc06-4efb-bf4a-b0559b38b1ed None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:15.485947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af06e25-fc06-4efb-bf4a-b0559b38b1ed None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:15.488790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 244/482] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:15.493007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472a3e4-fb6c-4fd4-97cd-b712933c784d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:15.493295 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472a3e4-fb6c-4fd4-97cd-b712933c784d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:15.493573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472a3e4-fb6c-4fd4-97cd-b712933c784d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:15.496962 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:15.524952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c472a3e4-fb6c-4fd4-97cd-b712933c784d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:15.526142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c472a3e4-fb6c-4fd4-97cd-b712933c784d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:15.530134 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 239/483] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:15.553179 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40066a45-747f-4a7b-8111-03b802afa8e2 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:15.654455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 245/484] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:36:15.701288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40066a45-747f-4a7b-8111-03b802afa8e2 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:15.702961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472a3e4-fb6c-4fd4-97cd-b712933c784d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:15.706797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:15.779430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a09a0-ac04-4231-b380-b0c957095fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:15.780307 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a09a0-ac04-4231-b380-b0c957095fc5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:15.780590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a09a0-ac04-4231-b380-b0c957095fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:15.780867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a09a0-ac04-4231-b380-b0c957095fc5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:15.802806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a09a0-ac04-4231-b380-b0c957095fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-1c6a09a0-ac04-4231-b380-b0c957095fc5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:15.812612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:15.813561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:15.814335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:15.814335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:15.814335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:15.814335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:15.814335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:15.814335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:15.814335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:15.814335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:15.814335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:15.814335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:15.814335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:15.814335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:15.815746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 240/485] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:15.821943 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a09a0-ac04-4231-b380-b0c957095fc5 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:15.880143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472a3e4-fb6c-4fd4-97cd-b712933c784d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:15.891370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:15.947173 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168d7a8-61ae-4ccb-81b2-eebe7c2d35af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:15.950298 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168d7a8-61ae-4ccb-81b2-eebe7c2d35af None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:15.950584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168d7a8-61ae-4ccb-81b2-eebe7c2d35af None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:15.950858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168d7a8-61ae-4ccb-81b2-eebe7c2d35af None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:15.981787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168d7a8-61ae-4ccb-81b2-eebe7c2d35af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-2168d7a8-61ae-4ccb-81b2-eebe7c2d35af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:15.989690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:15.990617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:15.991577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 246/486] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:15.999184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168d7a8-61ae-4ccb-81b2-eebe7c2d35af None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:16.024664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a09a0-ac04-4231-b380-b0c957095fc5 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:16.064880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a38a12-da71-43fa-8dd0-07e3cb8196e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:16.065549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a38a12-da71-43fa-8dd0-07e3cb8196e6 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.065833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a38a12-da71-43fa-8dd0-07e3cb8196e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.066110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a38a12-da71-43fa-8dd0-07e3cb8196e6 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.066860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a38a12-da71-43fa-8dd0-07e3cb8196e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:16.067168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a38a12-da71-43fa-8dd0-07e3cb8196e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:16.073186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a38a12-da71-43fa-8dd0-07e3cb8196e6 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:16.093669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 241/487] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:16.095020 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a38a12-da71-43fa-8dd0-07e3cb8196e6 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:16.136367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:16.202886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168d7a8-61ae-4ccb-81b2-eebe7c2d35af None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:16.251949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:16.263739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a38a12-da71-43fa-8dd0-07e3cb8196e6 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:16.278819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6f253-b00a-46f7-897a-fbdd587427c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:16.279854 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6f253-b00a-46f7-897a-fbdd587427c3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.280372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6f253-b00a-46f7-897a-fbdd587427c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.280839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6f253-b00a-46f7-897a-fbdd587427c3 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.281808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6f253-b00a-46f7-897a-fbdd587427c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:16.282322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6f253-b00a-46f7-897a-fbdd587427c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:16.283941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6f253-b00a-46f7-897a-fbdd587427c3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:16.307193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 247/488] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:16.313219 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6f253-b00a-46f7-897a-fbdd587427c3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:16.316549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4ee45-1b02-4460-b49b-f9cbc5005cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:16.317578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4ee45-1b02-4460-b49b-f9cbc5005cf6 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.318032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4ee45-1b02-4460-b49b-f9cbc5005cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.318618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4ee45-1b02-4460-b49b-f9cbc5005cf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.333853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4ee45-1b02-4460-b49b-f9cbc5005cf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:16.335698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4ee45-1b02-4460-b49b-f9cbc5005cf6 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-56c4ee45-1b02-4460-b49b-f9cbc5005cf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:16.357377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:16.358419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:16.359451 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:16.359451 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:16.359451 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:16.359451 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:16.361866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 242/489] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:16.363215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197681e-c2b3-47c1-87c5-d4bd723e1607 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.363652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197681e-c2b3-47c1-87c5-d4bd723e1607 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.364073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197681e-c2b3-47c1-87c5-d4bd723e1607 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.365808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 243/490] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:16.369073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197681e-c2b3-47c1-87c5-d4bd723e1607 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:16.449063 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6f253-b00a-46f7-897a-fbdd587427c3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:16.480037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:16.484372 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197681e-c2b3-47c1-87c5-d4bd723e1607 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:16.491251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805848a-f7ba-4247-852e-c6890858ad39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:16.492432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805848a-f7ba-4247-852e-c6890858ad39 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.493428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805848a-f7ba-4247-852e-c6890858ad39 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.493903 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805848a-f7ba-4247-852e-c6890858ad39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.507973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805848a-f7ba-4247-852e-c6890858ad39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:16.509422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805848a-f7ba-4247-852e-c6890858ad39 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:16.521961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70e3b0-5748-4b66-bc61-5a13fb77b4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:16.522626 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70e3b0-5748-4b66-bc61-5a13fb77b4c3 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.522908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70e3b0-5748-4b66-bc61-5a13fb77b4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.523183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70e3b0-5748-4b66-bc61-5a13fb77b4c3 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.523902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70e3b0-5748-4b66-bc61-5a13fb77b4c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:16.524237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70e3b0-5748-4b66-bc61-5a13fb77b4c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:16.525630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70e3b0-5748-4b66-bc61-5a13fb77b4c3 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-9805848a-f7ba-4247-852e-c6890858ad39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:16.529826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:16.530689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:16.537324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:16.537324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:16.537324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:16.537324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:16.537324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 248/491] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:16.537324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832db88-f491-45a8-a9b9-256b6308de9d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.537324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832db88-f491-45a8-a9b9-256b6308de9d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.537324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832db88-f491-45a8-a9b9-256b6308de9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.546585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 244/492] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:16.546585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f585e-884e-4a4c-9ab5-c9779c1af498 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.546585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f585e-884e-4a4c-9ab5-c9779c1af498 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.546585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f585e-884e-4a4c-9ab5-c9779c1af498 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.549669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 245/493] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:16.550304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f585e-884e-4a4c-9ab5-c9779c1af498 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:16.550400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e832db88-f491-45a8-a9b9-256b6308de9d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:16.551515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e832db88-f491-45a8-a9b9-256b6308de9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:16.562380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:16.621898 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 249/494] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:16.622910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832db88-f491-45a8-a9b9-256b6308de9d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:16.680499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f585e-884e-4a4c-9ab5-c9779c1af498 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:16.716487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9d663-f6ac-4db4-b3a6-e96b0510286a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:16.717168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9d663-f6ac-4db4-b3a6-e96b0510286a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.717449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9d663-f6ac-4db4-b3a6-e96b0510286a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.717719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9d663-f6ac-4db4-b3a6-e96b0510286a None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.718439 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9d663-f6ac-4db4-b3a6-e96b0510286a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:16.718742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9d663-f6ac-4db4-b3a6-e96b0510286a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:16.720148 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9d663-f6ac-4db4-b3a6-e96b0510286a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:16.731914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 246/495] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:16.733101 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4dab9-166b-4520-84f7-6daeb89d1ea5 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.733384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4dab9-166b-4520-84f7-6daeb89d1ea5 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.733661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4dab9-166b-4520-84f7-6daeb89d1ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.747674 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832db88-f491-45a8-a9b9-256b6308de9d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:16.749178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-41c4dab9-166b-4520-84f7-6daeb89d1ea5 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:16.750335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-41c4dab9-166b-4520-84f7-6daeb89d1ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:16.759599 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:16.786312 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381e0c8-2b1a-4290-a148-1d60a0f72e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:16.787284 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381e0c8-2b1a-4290-a148-1d60a0f72e82 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.787561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381e0c8-2b1a-4290-a148-1d60a0f72e82 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.787824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381e0c8-2b1a-4290-a148-1d60a0f72e82 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.795624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381e0c8-2b1a-4290-a148-1d60a0f72e82 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:16.797245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381e0c8-2b1a-4290-a148-1d60a0f72e82 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:16.811807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f381e0c8-2b1a-4290-a148-1d60a0f72e82 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:16.815454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:16.816395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:16.817272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:16.817272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:16.817272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:16.817272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:16.817272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:16.817272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:16.817272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:16.817272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:16.817272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 03:36:16.817272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:16.817272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 250/496] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:16.817857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381e0c8-2b1a-4290-a148-1d60a0f72e82 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:16.832178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 247/497] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:16.832178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4dab9-166b-4520-84f7-6daeb89d1ea5 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:16.929236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381e0c8-2b1a-4290-a148-1d60a0f72e82 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:16.941418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4dab9-166b-4520-84f7-6daeb89d1ea5 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:16.970879 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca389b-7385-45fd-99e8-8d450510b693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:16.971694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca389b-7385-45fd-99e8-8d450510b693 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.971984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca389b-7385-45fd-99e8-8d450510b693 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.972417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca389b-7385-45fd-99e8-8d450510b693 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.977413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb941df-7bba-4924-bb18-fabe17cb563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:16.978365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb941df-7bba-4924-bb18-fabe17cb563e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:16.978654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb941df-7bba-4924-bb18-fabe17cb563e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:16.978930 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb941df-7bba-4924-bb18-fabe17cb563e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:16.983543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca389b-7385-45fd-99e8-8d450510b693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:16.986193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb941df-7bba-4924-bb18-fabe17cb563e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:16.987659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb941df-7bba-4924-bb18-fabe17cb563e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-b2ca389b-7385-45fd-99e8-8d450510b693 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:16.991717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:16.992623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 251/498] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:16.998506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca389b-7385-45fd-99e8-8d450510b693 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-ceb941df-7bba-4924-bb18-fabe17cb563e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:17.004802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:17.005695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:17.006579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:17.006579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:17.006579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:17.006579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:17.006579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:17.006579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:17.006579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:17.006579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:17.006579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:36:17.006579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:17.008146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 248/499] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 12 03:36:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:17.009157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb941df-7bba-4924-bb18-fabe17cb563e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:17.025636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:17.065101 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:17.095202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca389b-7385-45fd-99e8-8d450510b693 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:17.106583 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb941df-7bba-4924-bb18-fabe17cb563e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:17.127520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ee079-a15f-4f34-ae31-2cce1fe2783d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:17.128205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ee079-a15f-4f34-ae31-2cce1fe2783d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:17.128486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ee079-a15f-4f34-ae31-2cce1fe2783d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:17.128752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ee079-a15f-4f34-ae31-2cce1fe2783d None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:17.129580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ee079-a15f-4f34-ae31-2cce1fe2783d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:17.129914 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ee079-a15f-4f34-ae31-2cce1fe2783d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:17.131423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ee079-a15f-4f34-ae31-2cce1fe2783d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:17.140889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd6f65-eecc-4425-8616-d210272e342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:17.141984 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd6f65-eecc-4425-8616-d210272e342d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:17.142265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd6f65-eecc-4425-8616-d210272e342d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:17.142529 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd6f65-eecc-4425-8616-d210272e342d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:17.153082 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd6f65-eecc-4425-8616-d210272e342d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-fcdd6f65-eecc-4425-8616-d210272e342d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:17.160812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:17.161911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:17.162731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 249/500] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:17.164109 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7f943-4892-41c7-827e-aa65eda54e5e None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:17.164109 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7f943-4892-41c7-827e-aa65eda54e5e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:17.164398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7f943-4892-41c7-827e-aa65eda54e5e None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:17.165844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 250/501] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:17.166444 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7f943-4892-41c7-827e-aa65eda54e5e None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:17.168620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 252/502] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Jul 12 03:36:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:17.169636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ee079-a15f-4f34-ae31-2cce1fe2783d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:17.256294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:17.270084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7f943-4892-41c7-827e-aa65eda54e5e None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:17.272237 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ee079-a15f-4f34-ae31-2cce1fe2783d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:17.287837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:17.304451 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662d649-6970-407a-9669-5e7a8694c887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:17.305144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662d649-6970-407a-9669-5e7a8694c887 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:17.305422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662d649-6970-407a-9669-5e7a8694c887 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:17.305693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662d649-6970-407a-9669-5e7a8694c887 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:17.306437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662d649-6970-407a-9669-5e7a8694c887 None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:17.306748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662d649-6970-407a-9669-5e7a8694c887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:17.308103 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac2253-64a5-4fd4-850b-d56252119528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:17.308237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662d649-6970-407a-9669-5e7a8694c887 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:17.308801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac2253-64a5-4fd4-850b-d56252119528 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:17.309078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac2253-64a5-4fd4-850b-d56252119528 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:17.309439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac2253-64a5-4fd4-850b-d56252119528 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:17.310398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac2253-64a5-4fd4-850b-d56252119528 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:17.310497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac2253-64a5-4fd4-850b-d56252119528 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:17.311898 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac2253-64a5-4fd4-850b-d56252119528 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:17.316420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 251/503] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:17.317219 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662d649-6970-407a-9669-5e7a8694c887 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:17.330976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 253/504] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 12 03:36:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:17.332307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703a354-2070-49e6-b133-db3b0be791e2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:17.332586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703a354-2070-49e6-b133-db3b0be791e2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:17.332870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703a354-2070-49e6-b133-db3b0be791e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:17.349824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4703a354-2070-49e6-b133-db3b0be791e2 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:17.351041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4703a354-2070-49e6-b133-db3b0be791e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:17.416405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 254/505] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:17.416561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662d649-6970-407a-9669-5e7a8694c887 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:17.417207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703a354-2070-49e6-b133-db3b0be791e2 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:17.452240 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bede33b-e2f6-4274-b3e0-0e1338c52ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:17.452900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bede33b-e2f6-4274-b3e0-0e1338c52ddc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:17.453189 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bede33b-e2f6-4274-b3e0-0e1338c52ddc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:17.453462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bede33b-e2f6-4274-b3e0-0e1338c52ddc None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:17.454192 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bede33b-e2f6-4274-b3e0-0e1338c52ddc None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:17.454485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bede33b-e2f6-4274-b3e0-0e1338c52ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:17.455863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bede33b-e2f6-4274-b3e0-0e1338c52ddc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:17.468131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 252/506] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:17.468131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bede33b-e2f6-4274-b3e0-0e1338c52ddc None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:17.505488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:17.506705 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:17.533633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703a354-2070-49e6-b133-db3b0be791e2 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:17.563781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bede33b-e2f6-4274-b3e0-0e1338c52ddc None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:17.568035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e4ee3-72ee-4c87-87a0-f3fa1681d850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:17.568685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e4ee3-72ee-4c87-87a0-f3fa1681d850 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:17.568968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e4ee3-72ee-4c87-87a0-f3fa1681d850 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:17.569241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e4ee3-72ee-4c87-87a0-f3fa1681d850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:17.596771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e651f92-3751-4d97-a085-56bf2d9fec87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:17.597565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e651f92-3751-4d97-a085-56bf2d9fec87 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:17.597838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e651f92-3751-4d97-a085-56bf2d9fec87 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:17.598122 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e651f92-3751-4d97-a085-56bf2d9fec87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:17.608374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e651f92-3751-4d97-a085-56bf2d9fec87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:17.609848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e651f92-3751-4d97-a085-56bf2d9fec87 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-0e651f92-3751-4d97-a085-56bf2d9fec87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:17.627470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:17.628381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:17.629207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:17.629207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:17.629207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:17.629207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:17.629207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 253/507] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:17.629795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e651f92-3751-4d97-a085-56bf2d9fec87 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:17.643422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e4ee3-72ee-4c87-87a0-f3fa1681d850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:17.645081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e4ee3-72ee-4c87-87a0-f3fa1681d850 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:17.717254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:17.725003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:17.731387 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e651f92-3751-4d97-a085-56bf2d9fec87 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:17.742713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 255/508] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:17.743753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e4ee3-72ee-4c87-87a0-f3fa1681d850 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:17.766369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b546d2-a63b-4aff-ad46-3f9c5281fec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:17.767568 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b546d2-a63b-4aff-ad46-3f9c5281fec2 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:17.767862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b546d2-a63b-4aff-ad46-3f9c5281fec2 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:17.768168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b546d2-a63b-4aff-ad46-3f9c5281fec2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:17.832728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b546d2-a63b-4aff-ad46-3f9c5281fec2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:17.834409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b546d2-a63b-4aff-ad46-3f9c5281fec2 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:17.842103 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e4ee3-72ee-4c87-87a0-f3fa1681d850 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:17.879183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07274fe-e8d6-4f5f-8970-518242b4e4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:17.880052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07274fe-e8d6-4f5f-8970-518242b4e4b7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:17.880347 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07274fe-e8d6-4f5f-8970-518242b4e4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:17.880619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07274fe-e8d6-4f5f-8970-518242b4e4b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:17.891318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07274fe-e8d6-4f5f-8970-518242b4e4b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:17.893200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07274fe-e8d6-4f5f-8970-518242b4e4b7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a07274fe-e8d6-4f5f-8970-518242b4e4b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:17.911091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:17.913236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:17.913236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:17.913236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:17.913236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:17.916340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 256/509] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:17.917497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07274fe-e8d6-4f5f-8970-518242b4e4b7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:17.931894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:17.939776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 254/510] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:17.940618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b546d2-a63b-4aff-ad46-3f9c5281fec2 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:17.947792 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:18.019321 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07274fe-e8d6-4f5f-8970-518242b4e4b7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:18.038158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b546d2-a63b-4aff-ad46-3f9c5281fec2 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:18.053041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077d163-7ae9-43cd-ae97-93c7290192fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:18.053842 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077d163-7ae9-43cd-ae97-93c7290192fe None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:18.054129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077d163-7ae9-43cd-ae97-93c7290192fe None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:18.054404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077d163-7ae9-43cd-ae97-93c7290192fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:18.064297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077d163-7ae9-43cd-ae97-93c7290192fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:18.065787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077d163-7ae9-43cd-ae97-93c7290192fe None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:18.070081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff4a9f-9efb-4f1f-9129-9b7c6bfc6847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:18.070764 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff4a9f-9efb-4f1f-9129-9b7c6bfc6847 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:18.071042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff4a9f-9efb-4f1f-9129-9b7c6bfc6847 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:18.071312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff4a9f-9efb-4f1f-9129-9b7c6bfc6847 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:18.072039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff4a9f-9efb-4f1f-9129-9b7c6bfc6847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:18.072349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff4a9f-9efb-4f1f-9129-9b7c6bfc6847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:18.073707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff4a9f-9efb-4f1f-9129-9b7c6bfc6847 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-1077d163-7ae9-43cd-ae97-93c7290192fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:18.083507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:18.084461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:18.085292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:18.085292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:18.085292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:18.085292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:18.085292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 257/511] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:18.085547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 255/512] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:18.085612 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077d163-7ae9-43cd-ae97-93c7290192fe None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:18.086213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff4a9f-9efb-4f1f-9129-9b7c6bfc6847 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:18.142347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:18.163251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:18.184507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077d163-7ae9-43cd-ae97-93c7290192fe None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:18.186767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff4a9f-9efb-4f1f-9129-9b7c6bfc6847 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:18.217111 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef9c79-e2fb-45d4-a17e-2d8410e7c3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:18.217899 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef9c79-e2fb-45d4-a17e-2d8410e7c3e0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:18.218185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef9c79-e2fb-45d4-a17e-2d8410e7c3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:18.218451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef9c79-e2fb-45d4-a17e-2d8410e7c3e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:18.219349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88a476-0ec0-4af4-910e-6a31f701957d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:18.220032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88a476-0ec0-4af4-910e-6a31f701957d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:18.220324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88a476-0ec0-4af4-910e-6a31f701957d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:18.220605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88a476-0ec0-4af4-910e-6a31f701957d None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:18.221337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88a476-0ec0-4af4-910e-6a31f701957d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:18.221648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88a476-0ec0-4af4-910e-6a31f701957d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:18.223031 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88a476-0ec0-4af4-910e-6a31f701957d None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:18.228358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef9c79-e2fb-45d4-a17e-2d8410e7c3e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:18.229827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef9c79-e2fb-45d4-a17e-2d8410e7c3e0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:18.234612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 256/513] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:18.235593 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88a476-0ec0-4af4-910e-6a31f701957d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-64ef9c79-e2fb-45d4-a17e-2d8410e7c3e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:18.247801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:18.248800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:18.249627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:18.249627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:18.249627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:18.249627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:18.249627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 258/514] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:18.249896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef9c79-e2fb-45d4-a17e-2d8410e7c3e0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:18.339893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88a476-0ec0-4af4-910e-6a31f701957d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:18.348245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef9c79-e2fb-45d4-a17e-2d8410e7c3e0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:18.354048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:18.373628 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea44a5-de3a-4f57-ad3e-528348f65535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:18.374443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea44a5-de3a-4f57-ad3e-528348f65535 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:18.374730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea44a5-de3a-4f57-ad3e-528348f65535 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:18.375000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea44a5-de3a-4f57-ad3e-528348f65535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:18.378725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:18.385254 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea44a5-de3a-4f57-ad3e-528348f65535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:18.386058 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cbfc4-c4b4-418d-8af6-0a66cfb88a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:18.386742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cbfc4-c4b4-418d-8af6-0a66cfb88a07 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:18.386834 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea44a5-de3a-4f57-ad3e-528348f65535 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:18.387030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cbfc4-c4b4-418d-8af6-0a66cfb88a07 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:18.387301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cbfc4-c4b4-418d-8af6-0a66cfb88a07 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:18.388073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cbfc4-c4b4-418d-8af6-0a66cfb88a07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:18.388383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cbfc4-c4b4-418d-8af6-0a66cfb88a07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:18.389762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cbfc4-c4b4-418d-8af6-0a66cfb88a07 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:18.401554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 259/515] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:18.402345 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cbfc4-c4b4-418d-8af6-0a66cfb88a07 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-10ea44a5-de3a-4f57-ad3e-528348f65535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:18.404318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:18.405187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:18.406045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:18.406045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:18.406045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:18.406045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:18.406045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 257/516] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:18.406315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea44a5-de3a-4f57-ad3e-528348f65535 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:18.499303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cbfc4-c4b4-418d-8af6-0a66cfb88a07 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:18.506938 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea44a5-de3a-4f57-ad3e-528348f65535 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:18.534321 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18c4df-f6cf-4127-88b7-56f75498977f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:18.535371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18c4df-f6cf-4127-88b7-56f75498977f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:18.535483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18c4df-f6cf-4127-88b7-56f75498977f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:18.535676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18c4df-f6cf-4127-88b7-56f75498977f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:18.539862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85684f6-9c02-4824-9a76-77ce9b5069e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:18.540571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85684f6-9c02-4824-9a76-77ce9b5069e1 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:18.540851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85684f6-9c02-4824-9a76-77ce9b5069e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:18.541128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85684f6-9c02-4824-9a76-77ce9b5069e1 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:18.541822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85684f6-9c02-4824-9a76-77ce9b5069e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:18.543188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18c4df-f6cf-4127-88b7-56f75498977f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:18.543301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85684f6-9c02-4824-9a76-77ce9b5069e1 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:18.544685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18c4df-f6cf-4127-88b7-56f75498977f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-2e18c4df-f6cf-4127-88b7-56f75498977f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:18.559253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:18.560221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:18.561143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:18.561143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:18.561143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:18.561143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:18.561143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:18.561143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:18.561143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:18.561143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:18.561143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 03:36:18.561143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:18.561143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 260/517] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:18.561699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18c4df-f6cf-4127-88b7-56f75498977f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:18.574590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 258/518] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:36:18 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:36:18.574590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85684f6-9c02-4824-9a76-77ce9b5069e1 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:18.602913 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:18.621553 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:18.687653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18c4df-f6cf-4127-88b7-56f75498977f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:18.690729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85684f6-9c02-4824-9a76-77ce9b5069e1 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:18.731861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c54fb-e245-40e9-9ba7-5c350abda363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:18.732825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c54fb-e245-40e9-9ba7-5c350abda363 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:18.733235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df552267-f1bd-4d9c-851e-ad4b32cd2dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:18.733352 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c54fb-e245-40e9-9ba7-5c350abda363 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:18.733825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c54fb-e245-40e9-9ba7-5c350abda363 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:18.733982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df552267-f1bd-4d9c-851e-ad4b32cd2dcd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:18.734227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df552267-f1bd-4d9c-851e-ad4b32cd2dcd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:18.734445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df552267-f1bd-4d9c-851e-ad4b32cd2dcd None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:18.735175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df552267-f1bd-4d9c-851e-ad4b32cd2dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:18.735475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df552267-f1bd-4d9c-851e-ad4b32cd2dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:18.737654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df552267-f1bd-4d9c-851e-ad4b32cd2dcd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:18.746816 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c54fb-e245-40e9-9ba7-5c350abda363 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:18.748868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c54fb-e245-40e9-9ba7-5c350abda363 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:18.753309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 261/519] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:18.754045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df552267-f1bd-4d9c-851e-ad4b32cd2dcd None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-262c54fb-e245-40e9-9ba7-5c350abda363 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:18.770780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:18.771749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:18.772688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:18.772688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:18.772688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:18.772688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:18.772688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:18.772688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:18.772688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:18.772688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:18.772688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 03:36:18.772688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:18.773217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 259/520] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:18.774143 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c54fb-e245-40e9-9ba7-5c350abda363 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:18.860972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:18.862332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df552267-f1bd-4d9c-851e-ad4b32cd2dcd None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:18.874651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:18.880572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c54fb-e245-40e9-9ba7-5c350abda363 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:18.902399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609740f-bfb5-475a-bb93-e964fe84313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:18.903068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609740f-bfb5-475a-bb93-e964fe84313b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:18.903351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609740f-bfb5-475a-bb93-e964fe84313b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:18.903627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609740f-bfb5-475a-bb93-e964fe84313b None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:18.904381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609740f-bfb5-475a-bb93-e964fe84313b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:18.904893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609740f-bfb5-475a-bb93-e964fe84313b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:18.906084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609740f-bfb5-475a-bb93-e964fe84313b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:18.915727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a8959-d4ca-43df-87d7-bb4b598112eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:18.916449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a8959-d4ca-43df-87d7-bb4b598112eb None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:18.916726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a8959-d4ca-43df-87d7-bb4b598112eb None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:18.916991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a8959-d4ca-43df-87d7-bb4b598112eb None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:18.917809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a8959-d4ca-43df-87d7-bb4b598112eb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:18.918336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a8959-d4ca-43df-87d7-bb4b598112eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:18.918434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 262/521] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:18.919716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a8959-d4ca-43df-87d7-bb4b598112eb None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:18.920960 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609740f-bfb5-475a-bb93-e964fe84313b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:18.967793 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 260/522] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Jul 12 03:36:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:18.967793 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a8959-d4ca-43df-87d7-bb4b598112eb None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:19.041458 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609740f-bfb5-475a-bb93-e964fe84313b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:19.073113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a8959-d4ca-43df-87d7-bb4b598112eb None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:19.080832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b84ea4-27e6-4fda-83f4-adad0d96cd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:19.081828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b84ea4-27e6-4fda-83f4-adad0d96cd76 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:19.082081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b84ea4-27e6-4fda-83f4-adad0d96cd76 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:19.082385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b84ea4-27e6-4fda-83f4-adad0d96cd76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:19.090143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b84ea4-27e6-4fda-83f4-adad0d96cd76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:19.091791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b84ea4-27e6-4fda-83f4-adad0d96cd76 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-b2b84ea4-27e6-4fda-83f4-adad0d96cd76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:19.107449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:19.108380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:19.109242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:19.109242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:19.109242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:19.109242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:19.109242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:19.109242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:19.109242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:19.109242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:19.109242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 263/523] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:19.109828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b84ea4-27e6-4fda-83f4-adad0d96cd76 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:19.120572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f532dc-ef1e-406c-94f0-e7563553ed29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:19.121195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f532dc-ef1e-406c-94f0-e7563553ed29 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:19.121833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f532dc-ef1e-406c-94f0-e7563553ed29 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:19.122115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f532dc-ef1e-406c-94f0-e7563553ed29 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:19.123301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f532dc-ef1e-406c-94f0-e7563553ed29 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:19.123768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f532dc-ef1e-406c-94f0-e7563553ed29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:19.125270 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f532dc-ef1e-406c-94f0-e7563553ed29 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:19.134707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:19.143751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:19.148131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 261/524] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Jul 12 03:36:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:19.148131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f532dc-ef1e-406c-94f0-e7563553ed29 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:19.225371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b84ea4-27e6-4fda-83f4-adad0d96cd76 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:19.250168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f532dc-ef1e-406c-94f0-e7563553ed29 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:19.265481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513835c-91b0-4774-b13b-c395472efcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:19.266715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513835c-91b0-4774-b13b-c395472efcfd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:19.266993 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513835c-91b0-4774-b13b-c395472efcfd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:19.267246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513835c-91b0-4774-b13b-c395472efcfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:19.274388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513835c-91b0-4774-b13b-c395472efcfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:19.275911 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513835c-91b0-4774-b13b-c395472efcfd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:19.284899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0b17e-a34c-4ac7-ba8f-ee187b830a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:19.285568 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0b17e-a34c-4ac7-ba8f-ee187b830a15 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:19.285878 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0b17e-a34c-4ac7-ba8f-ee187b830a15 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:19.286164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0b17e-a34c-4ac7-ba8f-ee187b830a15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-7513835c-91b0-4774-b13b-c395472efcfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:19.290500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:19.291352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:19.297300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:19.297300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:19.297300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:19.297300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:19.297300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:19.297300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:19.297300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:19.297300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:19.297300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 264/525] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:19.297300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513835c-91b0-4774-b13b-c395472efcfd None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:19.357162 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:19.362104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0b17e-a34c-4ac7-ba8f-ee187b830a15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:19.363592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0b17e-a34c-4ac7-ba8f-ee187b830a15 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:19.394509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:19.405392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513835c-91b0-4774-b13b-c395472efcfd None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:19.440731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7def2f-6edf-49c5-84e4-4c866859caac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:19.441446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7def2f-6edf-49c5-84e4-4c866859caac None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:19.441732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7def2f-6edf-49c5-84e4-4c866859caac None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:19.442022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7def2f-6edf-49c5-84e4-4c866859caac None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:19.442844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7def2f-6edf-49c5-84e4-4c866859caac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:19.443149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7def2f-6edf-49c5-84e4-4c866859caac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:19.444688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7def2f-6edf-49c5-84e4-4c866859caac None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:19.462558 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 265/526] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:19.463242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7def2f-6edf-49c5-84e4-4c866859caac None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:19.481456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 262/527] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:19.481456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0b17e-a34c-4ac7-ba8f-ee187b830a15 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:19.572228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7def2f-6edf-49c5-84e4-4c866859caac None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:19.575300 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:19.579559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0b17e-a34c-4ac7-ba8f-ee187b830a15 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:19.608039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabda8d-7fa0-41fa-af3f-637ce47626b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:19.608684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabda8d-7fa0-41fa-af3f-637ce47626b6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:19.608966 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabda8d-7fa0-41fa-af3f-637ce47626b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:19.609243 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabda8d-7fa0-41fa-af3f-637ce47626b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:19.614412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22847de5-f30d-49f8-9693-a969242a872f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:19.615071 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22847de5-f30d-49f8-9693-a969242a872f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:19.615356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22847de5-f30d-49f8-9693-a969242a872f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:19.615630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22847de5-f30d-49f8-9693-a969242a872f None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:19.616442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22847de5-f30d-49f8-9693-a969242a872f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:19.616752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22847de5-f30d-49f8-9693-a969242a872f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:19.618190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22847de5-f30d-49f8-9693-a969242a872f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:19.628784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:19.637433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 263/528] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:19.643159 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22847de5-f30d-49f8-9693-a969242a872f None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:19.682908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabda8d-7fa0-41fa-af3f-637ce47626b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:19.684350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabda8d-7fa0-41fa-af3f-637ce47626b6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:19.745992 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22847de5-f30d-49f8-9693-a969242a872f None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:19.782258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961f062-a97c-4bd3-8165-76bf65de1cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:19.783631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961f062-a97c-4bd3-8165-76bf65de1cf2 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:19.784030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961f062-a97c-4bd3-8165-76bf65de1cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:19.784418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961f062-a97c-4bd3-8165-76bf65de1cf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:19.785385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961f062-a97c-4bd3-8165-76bf65de1cf2 None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=7297224dc73c4358b60f1b096c0fb5bf) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:19.785893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961f062-a97c-4bd3-8165-76bf65de1cf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:19.787465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961f062-a97c-4bd3-8165-76bf65de1cf2 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:19.792886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 266/529] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:19.792886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabda8d-7fa0-41fa-af3f-637ce47626b6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:19.804034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 264/530] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:19 2022] GET /identity/v3/role_assignments?user.id=7297224dc73c4358b60f1b096c0fb5bf&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:19.804034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961f062-a97c-4bd3-8165-76bf65de1cf2 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:19.844095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:19.844510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:19.922377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabda8d-7fa0-41fa-af3f-637ce47626b6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:19.937298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961f062-a97c-4bd3-8165-76bf65de1cf2 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:19.971994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb542f-01b4-4546-8449-957f6fc21172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:19.973066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb542f-01b4-4546-8449-957f6fc21172 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:19.973488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb542f-01b4-4546-8449-957f6fc21172 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:19.973858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb542f-01b4-4546-8449-957f6fc21172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:19.981014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bdfa9-9f54-4fd0-bf2f-9d5e24536f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:19.981957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bdfa9-9f54-4fd0-bf2f-9d5e24536f4f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:19.982397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bdfa9-9f54-4fd0-bf2f-9d5e24536f4f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:19.982805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bdfa9-9f54-4fd0-bf2f-9d5e24536f4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:19.983874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bdfa9-9f54-4fd0-bf2f-9d5e24536f4f None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=7138318e53ff4b7885942d49b19eb0a5) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:19.984364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bdfa9-9f54-4fd0-bf2f-9d5e24536f4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:19.985919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb542f-01b4-4546-8449-957f6fc21172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:19.986140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bdfa9-9f54-4fd0-bf2f-9d5e24536f4f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:19.987566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb542f-01b4-4546-8449-957f6fc21172 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:19.992663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 265/531] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:19 2022] GET /identity/v3/role_assignments?user.id=7138318e53ff4b7885942d49b19eb0a5&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:19.993655 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bdfa9-9f54-4fd0-bf2f-9d5e24536f4f None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-fecb542f-01b4-4546-8449-957f6fc21172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:20.027164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:20.028336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:20.029363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:20.029363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:20.029363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:20.029363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:20.039832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 267/532] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:20.042986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb542f-01b4-4546-8449-957f6fc21172 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:20.186971 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:20.205583 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bdfa9-9f54-4fd0-bf2f-9d5e24536f4f None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:20.227820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb542f-01b4-4546-8449-957f6fc21172 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:20.229566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:20.246588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c600836-797e-4161-8e4f-373094fa3a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:20.247836 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c600836-797e-4161-8e4f-373094fa3a69 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:20.248281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c600836-797e-4161-8e4f-373094fa3a69 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:20.248687 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c600836-797e-4161-8e4f-373094fa3a69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:20.264101 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c600836-797e-4161-8e4f-373094fa3a69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:20.267787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c600836-797e-4161-8e4f-373094fa3a69 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-6c600836-797e-4161-8e4f-373094fa3a69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:20.298748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:20.299805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:20.300883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:20.300883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:20.300883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:20.300883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:20.302586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 266/533] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:20.303430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9056d15-3481-4bda-bac3-daf42bf411ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:20.308512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9056d15-3481-4bda-bac3-daf42bf411ee None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:20.308512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9056d15-3481-4bda-bac3-daf42bf411ee None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:20.308512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9056d15-3481-4bda-bac3-daf42bf411ee None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:20.308512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9056d15-3481-4bda-bac3-daf42bf411ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:20.308512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9056d15-3481-4bda-bac3-daf42bf411ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:20.308512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9056d15-3481-4bda-bac3-daf42bf411ee None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:20.310041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c600836-797e-4161-8e4f-373094fa3a69 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:20.319854 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 268/534] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:20.329005 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9056d15-3481-4bda-bac3-daf42bf411ee None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:20.429718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c600836-797e-4161-8e4f-373094fa3a69 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:20.474951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3f47a-68f7-41fa-8a55-fde6ab2648d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:20.475568 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3f47a-68f7-41fa-8a55-fde6ab2648d2 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:20.475910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3f47a-68f7-41fa-8a55-fde6ab2648d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:20.476219 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3f47a-68f7-41fa-8a55-fde6ab2648d2 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:20.477070 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3f47a-68f7-41fa-8a55-fde6ab2648d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:20.477391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3f47a-68f7-41fa-8a55-fde6ab2648d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:20.478800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3f47a-68f7-41fa-8a55-fde6ab2648d2 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:20.496321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:20.502646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 267/535] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:20.505440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9056d15-3481-4bda-bac3-daf42bf411ee None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:20.507850 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3f47a-68f7-41fa-8a55-fde6ab2648d2 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:20.565435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603c756-864a-4d77-aac7-decbf3affbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:20.566804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603c756-864a-4d77-aac7-decbf3affbaf None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:20.569098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603c756-864a-4d77-aac7-decbf3affbaf None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:20.569568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603c756-864a-4d77-aac7-decbf3affbaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:20.586299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603c756-864a-4d77-aac7-decbf3affbaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:20.588765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603c756-864a-4d77-aac7-decbf3affbaf None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-7603c756-864a-4d77-aac7-decbf3affbaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:20.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:20.614212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:20.615083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:20.615083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:20.615083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:20.615083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:20.615083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:20.615083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:20.615083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:20.615083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:20.628368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 269/536] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:20.634692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603c756-864a-4d77-aac7-decbf3affbaf None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:20.679401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:20.695186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3f47a-68f7-41fa-8a55-fde6ab2648d2 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:20.747690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24612c0e-8848-44f6-a48e-81964f6d4134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:20.748441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24612c0e-8848-44f6-a48e-81964f6d4134 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:20.748731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24612c0e-8848-44f6-a48e-81964f6d4134 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:20.749009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24612c0e-8848-44f6-a48e-81964f6d4134 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:20.758741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24612c0e-8848-44f6-a48e-81964f6d4134 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:20.760515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24612c0e-8848-44f6-a48e-81964f6d4134 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:20.777248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603c756-864a-4d77-aac7-decbf3affbaf None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-24612c0e-8848-44f6-a48e-81964f6d4134 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:20.781924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:20.782788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:20.783607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:20.783607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:20.783607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:20.783607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:20.783607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:20.783607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:20.783607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:20.783607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:20.784588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 268/537] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:20.785560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd5a8a-b414-41ca-b4ee-db9316be8ecd None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:20.786078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd5a8a-b414-41ca-b4ee-db9316be8ecd None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:20.786542 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd5a8a-b414-41ca-b4ee-db9316be8ecd None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:20.792072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 269/538] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:20.793034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd5a8a-b414-41ca-b4ee-db9316be8ecd None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:20.833486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b96a8e-bd98-4b2c-b1de-e7c145b35a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:20.834201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b96a8e-bd98-4b2c-b1de-e7c145b35a02 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:20.834540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b96a8e-bd98-4b2c-b1de-e7c145b35a02 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:20.834816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b96a8e-bd98-4b2c-b1de-e7c145b35a02 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:20.835652 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b96a8e-bd98-4b2c-b1de-e7c145b35a02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:20.835970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b96a8e-bd98-4b2c-b1de-e7c145b35a02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:20.837533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b96a8e-bd98-4b2c-b1de-e7c145b35a02 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:20.854487 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 270/539] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:20.856338 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da78487-b7e3-432a-a0e0-e87d0c33f85f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:20.856628 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da78487-b7e3-432a-a0e0-e87d0c33f85f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:20.857065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da78487-b7e3-432a-a0e0-e87d0c33f85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:20.858702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:20.880435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8da78487-b7e3-432a-a0e0-e87d0c33f85f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:20.882359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8da78487-b7e3-432a-a0e0-e87d0c33f85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:20.906161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd5a8a-b414-41ca-b4ee-db9316be8ecd None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:20.951731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c5fac-f329-48c2-be2d-8d7a9cdf41fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:20.953306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c5fac-f329-48c2-be2d-8d7a9cdf41fc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:20.953808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c5fac-f329-48c2-be2d-8d7a9cdf41fc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:20.954487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c5fac-f329-48c2-be2d-8d7a9cdf41fc None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:20.955860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c5fac-f329-48c2-be2d-8d7a9cdf41fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:20.956759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c5fac-f329-48c2-be2d-8d7a9cdf41fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:20.958934 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c5fac-f329-48c2-be2d-8d7a9cdf41fc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:20.972081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 271/540] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:20.973344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da78487-b7e3-432a-a0e0-e87d0c33f85f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:20.983835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 270/541] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:20.988331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c5fac-f329-48c2-be2d-8d7a9cdf41fc None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:21.000268 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:21.114059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da78487-b7e3-432a-a0e0-e87d0c33f85f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:21.115735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c5fac-f329-48c2-be2d-8d7a9cdf41fc None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:21.137043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:21.158529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962293c-b33e-419c-9c20-a46c55c63415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:21.159260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c5c9f-9616-430c-a618-1a1b6f00630a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:21.159385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962293c-b33e-419c-9c20-a46c55c63415 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:21.159645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962293c-b33e-419c-9c20-a46c55c63415 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:21.159990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962293c-b33e-419c-9c20-a46c55c63415 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:21.160699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c5c9f-9616-430c-a618-1a1b6f00630a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:21.161071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962293c-b33e-419c-9c20-a46c55c63415 None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=139be21e7f644b5796d30158851ebd00) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:21.161332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c5c9f-9616-430c-a618-1a1b6f00630a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:21.161468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962293c-b33e-419c-9c20-a46c55c63415 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:21.161886 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c5c9f-9616-430c-a618-1a1b6f00630a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:21.162881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c5c9f-9616-430c-a618-1a1b6f00630a None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=cee6da907b614dc5b6ffb47e2860f459) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:21.163156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962293c-b33e-419c-9c20-a46c55c63415 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:21.163534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c5c9f-9616-430c-a618-1a1b6f00630a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:21.165321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c5c9f-9616-430c-a618-1a1b6f00630a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:21.170240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 272/542] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:20 2022] GET /identity/v3/role_assignments?user.id=139be21e7f644b5796d30158851ebd00&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:21.171323 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962293c-b33e-419c-9c20-a46c55c63415 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:21.172890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 271/543] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:20 2022] GET /identity/v3/role_assignments?user.id=cee6da907b614dc5b6ffb47e2860f459&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:21.267086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:21.296137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962293c-b33e-419c-9c20-a46c55c63415 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:21.340166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f265e0-55de-4e88-a76b-5b5044a4207e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:21.341130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f265e0-55de-4e88-a76b-5b5044a4207e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:21.341415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f265e0-55de-4e88-a76b-5b5044a4207e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:21.341689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f265e0-55de-4e88-a76b-5b5044a4207e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:21.366411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f265e0-55de-4e88-a76b-5b5044a4207e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-61f265e0-55de-4e88-a76b-5b5044a4207e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:21.374555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:21.375529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:21.376393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:21.376393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:21.376393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:21.376393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:21.376393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:21.376393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:21.376393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:21.376393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:21.376393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:21.376393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:21.376393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:21.376393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:21.380337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 273/544] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:21.382472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c5c9f-9616-430c-a618-1a1b6f00630a None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:21.527145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c5c9f-9616-430c-a618-1a1b6f00630a None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:21.562711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ead08f-3db0-475d-93ca-096379d8539b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:21.563806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ead08f-3db0-475d-93ca-096379d8539b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:21.564326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ead08f-3db0-475d-93ca-096379d8539b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:21.564786 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ead08f-3db0-475d-93ca-096379d8539b None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:21.565715 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ead08f-3db0-475d-93ca-096379d8539b None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:21.566222 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ead08f-3db0-475d-93ca-096379d8539b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:21.567813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ead08f-3db0-475d-93ca-096379d8539b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:21.576861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 272/545] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:21.588350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f265e0-55de-4e88-a76b-5b5044a4207e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:21.705247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f265e0-55de-4e88-a76b-5b5044a4207e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:21.750252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c6db-045c-4fd0-9e91-97edfa9fd1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:21.751476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c6db-045c-4fd0-9e91-97edfa9fd1b4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:21.751961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c6db-045c-4fd0-9e91-97edfa9fd1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:21.752449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c6db-045c-4fd0-9e91-97edfa9fd1b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:21.757447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:21.767687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76c6db-045c-4fd0-9e91-97edfa9fd1b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:21.769462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76c6db-045c-4fd0-9e91-97edfa9fd1b4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6e76c6db-045c-4fd0-9e91-97edfa9fd1b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:21.790965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:21.791833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:21.793299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:21.793299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:21.793299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:21.793299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:21.793299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 274/546] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:21.797847 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ead08f-3db0-475d-93ca-096379d8539b None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:21.862820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:21.932416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ead08f-3db0-475d-93ca-096379d8539b None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:21.993758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86960005-46df-4277-8279-5fa9c2ae9657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:21.994429 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86960005-46df-4277-8279-5fa9c2ae9657 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:21.994710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86960005-46df-4277-8279-5fa9c2ae9657 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:21.994980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86960005-46df-4277-8279-5fa9c2ae9657 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:21.995698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86960005-46df-4277-8279-5fa9c2ae9657 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:21.996015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86960005-46df-4277-8279-5fa9c2ae9657 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:21.997402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86960005-46df-4277-8279-5fa9c2ae9657 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:22.015220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 273/547] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:22.021106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c6db-045c-4fd0-9e91-97edfa9fd1b4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:22.046900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258ebc9-6c2e-476d-b2bd-48a1860448ae None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:22.047199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258ebc9-6c2e-476d-b2bd-48a1860448ae None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:22.047391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258ebc9-6c2e-476d-b2bd-48a1860448ae None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:22.048779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 274/548] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:22.060896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7735de5-431a-4de9-9f09-4526b89a3f01 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:22.061314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7735de5-431a-4de9-9f09-4526b89a3f01 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:22.061628 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7735de5-431a-4de9-9f09-4526b89a3f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:22.089500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d7735de5-431a-4de9-9f09-4526b89a3f01 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:22.090729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d7735de5-431a-4de9-9f09-4526b89a3f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:22.214380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 275/549] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 03:36:22.218459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c6db-045c-4fd0-9e91-97edfa9fd1b4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:22.249005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7735de5-431a-4de9-9f09-4526b89a3f01 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:22.265777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104c0c9-fbc6-47e5-80f6-7288e8922f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:22.266646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104c0c9-fbc6-47e5-80f6-7288e8922f98 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:22.266931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104c0c9-fbc6-47e5-80f6-7288e8922f98 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:22.267199 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104c0c9-fbc6-47e5-80f6-7288e8922f98 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:22.283233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104c0c9-fbc6-47e5-80f6-7288e8922f98 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:22.284725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104c0c9-fbc6-47e5-80f6-7288e8922f98 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:22.310710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-2104c0c9-fbc6-47e5-80f6-7288e8922f98 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:22.313106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:22.314687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:22.315777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:22.315777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:22.315777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:22.315777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:22.315777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:22.315777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:22.315777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:22.315777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:22.315777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 03:36:22.315777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:22.315777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 275/550] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:22.325486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104c0c9-fbc6-47e5-80f6-7288e8922f98 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:22.414510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7735de5-431a-4de9-9f09-4526b89a3f01 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:22.428877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:22.453612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be123e-9abe-4506-ad9d-80b25b277e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:22.454417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be123e-9abe-4506-ad9d-80b25b277e6c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:22.454702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be123e-9abe-4506-ad9d-80b25b277e6c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:22.454979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be123e-9abe-4506-ad9d-80b25b277e6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:22.465941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be123e-9abe-4506-ad9d-80b25b277e6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-b8be123e-9abe-4506-ad9d-80b25b277e6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:22.473691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:22.474599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:22.475425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 276/551] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:22.476833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a53fc-6ece-4f6e-afba-7116aea24572 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:22.476833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a53fc-6ece-4f6e-afba-7116aea24572 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:22.477120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a53fc-6ece-4f6e-afba-7116aea24572 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:22.478612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 277/552] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:22.480544 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a53fc-6ece-4f6e-afba-7116aea24572 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:22.492863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104c0c9-fbc6-47e5-80f6-7288e8922f98 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:22.529193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdf3e8-58d3-407e-a7ba-0cf6e20d48a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:22.529985 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdf3e8-58d3-407e-a7ba-0cf6e20d48a4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:22.530267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdf3e8-58d3-407e-a7ba-0cf6e20d48a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:22.530539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdf3e8-58d3-407e-a7ba-0cf6e20d48a4 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:22.531373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdf3e8-58d3-407e-a7ba-0cf6e20d48a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:22.531676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdf3e8-58d3-407e-a7ba-0cf6e20d48a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:22.533964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdf3e8-58d3-407e-a7ba-0cf6e20d48a4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:22.554708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 276/553] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Jul 12 03:36:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:22.556123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fa797-9f62-4056-a679-5302da3ca1b8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:22.556409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fa797-9f62-4056-a679-5302da3ca1b8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:22.556777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fa797-9f62-4056-a679-5302da3ca1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:22.574215 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-cd3fa797-9f62-4056-a679-5302da3ca1b8 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:22.575330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cd3fa797-9f62-4056-a679-5302da3ca1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:22.583539 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:22.592501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a53fc-6ece-4f6e-afba-7116aea24572 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:22.632694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775cea4-379c-406c-ba6c-293fb47fa763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:22.633414 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775cea4-379c-406c-ba6c-293fb47fa763 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:22.634152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775cea4-379c-406c-ba6c-293fb47fa763 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:22.634152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775cea4-379c-406c-ba6c-293fb47fa763 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:22.634889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775cea4-379c-406c-ba6c-293fb47fa763 None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:22.635200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775cea4-379c-406c-ba6c-293fb47fa763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:22.636682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775cea4-379c-406c-ba6c-293fb47fa763 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:22.646888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 278/554] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:22.647906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775cea4-379c-406c-ba6c-293fb47fa763 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:22.662457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 277/555] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:22.664204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fa797-9f62-4056-a679-5302da3ca1b8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:22.717336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:22.754707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775cea4-379c-406c-ba6c-293fb47fa763 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:22.786351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fa797-9f62-4056-a679-5302da3ca1b8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:22.791288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889affd9-fc7a-492e-a387-98cb9339a3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:22.792155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889affd9-fc7a-492e-a387-98cb9339a3df None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:22.792599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889affd9-fc7a-492e-a387-98cb9339a3df None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:22.793014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889affd9-fc7a-492e-a387-98cb9339a3df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:22.819316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:22.822990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5519cf-6da1-49c0-a20b-c21be882e6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:22.824037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5519cf-6da1-49c0-a20b-c21be882e6ac None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:22.824974 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5519cf-6da1-49c0-a20b-c21be882e6ac None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:22.825814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5519cf-6da1-49c0-a20b-c21be882e6ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:22.837107 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5519cf-6da1-49c0-a20b-c21be882e6ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:22.838825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5519cf-6da1-49c0-a20b-c21be882e6ac None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-8b5519cf-6da1-49c0-a20b-c21be882e6ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:22.856826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:22.857851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:22.858820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:22.858820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:22.858820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:22.858820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:22.860720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 278/556] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:22.861762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5519cf-6da1-49c0-a20b-c21be882e6ac None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:22.863461 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889affd9-fc7a-492e-a387-98cb9339a3df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:22.865055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889affd9-fc7a-492e-a387-98cb9339a3df None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:22.953985 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:22.968498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5519cf-6da1-49c0-a20b-c21be882e6ac None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:22.977669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 279/557] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:22.978921 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889affd9-fc7a-492e-a387-98cb9339a3df None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:23.004202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76080b67-d229-4cf5-8ce3-befed7dc2165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:23.005011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76080b67-d229-4cf5-8ce3-befed7dc2165 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.005289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76080b67-d229-4cf5-8ce3-befed7dc2165 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.005559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76080b67-d229-4cf5-8ce3-befed7dc2165 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.016903 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76080b67-d229-4cf5-8ce3-befed7dc2165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-76080b67-d229-4cf5-8ce3-befed7dc2165 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:23.025216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:23.026634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:23.027442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:23.027442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:23.027442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:23.027442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:23.027442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:23.027442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:23.027442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:23.027442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:23.027442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:23.027442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:23.027442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:23.027442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:23.031455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 279/558] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:23.032432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76080b67-d229-4cf5-8ce3-befed7dc2165 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:23.040151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:23.087350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889affd9-fc7a-492e-a387-98cb9339a3df None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:23.123910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669b867-33c6-4c37-8aba-543a0c27d623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:23.124619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669b867-33c6-4c37-8aba-543a0c27d623 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.124910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669b867-33c6-4c37-8aba-543a0c27d623 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.125190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669b867-33c6-4c37-8aba-543a0c27d623 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.126022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669b867-33c6-4c37-8aba-543a0c27d623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:23.126337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669b867-33c6-4c37-8aba-543a0c27d623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:23.127799 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669b867-33c6-4c37-8aba-543a0c27d623 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:23.141341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 280/559] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:23.142270 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669b867-33c6-4c37-8aba-543a0c27d623 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:23.143098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76080b67-d229-4cf5-8ce3-befed7dc2165 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:23.179242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e93a0-1fc2-45d4-b2ad-7550eb02c9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:23.180405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e93a0-1fc2-45d4-b2ad-7550eb02c9e2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.180824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e93a0-1fc2-45d4-b2ad-7550eb02c9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.181255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e93a0-1fc2-45d4-b2ad-7550eb02c9e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.192980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e93a0-1fc2-45d4-b2ad-7550eb02c9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:23.194748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e93a0-1fc2-45d4-b2ad-7550eb02c9e2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:23.203669 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-932e93a0-1fc2-45d4-b2ad-7550eb02c9e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:23.216550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:23.217738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:23.218760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:23.218760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:23.218760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:23.218760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:23.222433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 280/560] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:23.223500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e93a0-1fc2-45d4-b2ad-7550eb02c9e2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:23.258671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669b867-33c6-4c37-8aba-543a0c27d623 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:23.277213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:23.297216 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64e5b1-6a28-4c27-88cf-1573f4c01862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:23.298412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64e5b1-6a28-4c27-88cf-1573f4c01862 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.298873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64e5b1-6a28-4c27-88cf-1573f4c01862 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.299372 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64e5b1-6a28-4c27-88cf-1573f4c01862 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.300329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64e5b1-6a28-4c27-88cf-1573f4c01862 None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:23.300869 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64e5b1-6a28-4c27-88cf-1573f4c01862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:23.302612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e64e5b1-6a28-4c27-88cf-1573f4c01862 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:23.311510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 281/561] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:23.312621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64e5b1-6a28-4c27-88cf-1573f4c01862 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:23.339386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e93a0-1fc2-45d4-b2ad-7550eb02c9e2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:23.375515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b44a78-8966-412b-a189-6a156d641bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:23.376351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b44a78-8966-412b-a189-6a156d641bee None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.376732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b44a78-8966-412b-a189-6a156d641bee None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.376994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b44a78-8966-412b-a189-6a156d641bee None admin] PATH_INFO: `/v3/users/glance` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.386405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b44a78-8966-412b-a189-6a156d641bee None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:23.387894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b44a78-8966-412b-a189-6a156d641bee None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-78b44a78-8966-412b-a189-6a156d641bee None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:23.403554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:23.404641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:23.405564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:23.405564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:23.405564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:23.405564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:23.405564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:23.405564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:23.405564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:23.405564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:23.405564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 03:36:23.405564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:23.405564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 281/562] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:23.406164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b44a78-8966-412b-a189-6a156d641bee None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:23.426093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64e5b1-6a28-4c27-88cf-1573f4c01862 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:23.463436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d100f4-8a82-4313-99f7-f0fc917f75f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:23.464486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d100f4-8a82-4313-99f7-f0fc917f75f7 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.464918 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d100f4-8a82-4313-99f7-f0fc917f75f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.465426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d100f4-8a82-4313-99f7-f0fc917f75f7 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.466457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d100f4-8a82-4313-99f7-f0fc917f75f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:23.466910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d100f4-8a82-4313-99f7-f0fc917f75f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:23.468778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d100f4-8a82-4313-99f7-f0fc917f75f7 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:23.469209 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:23.484510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 282/563] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:23.485642 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d100f4-8a82-4313-99f7-f0fc917f75f7 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:23.519589 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b44a78-8966-412b-a189-6a156d641bee None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:23.531047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:23.556853 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0630b-e5ee-4fe1-9ec7-4c99eb67b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:23.558028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0630b-e5ee-4fe1-9ec7-4c99eb67b25a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.558368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0630b-e5ee-4fe1-9ec7-4c99eb67b25a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.558651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0630b-e5ee-4fe1-9ec7-4c99eb67b25a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.570909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0630b-e5ee-4fe1-9ec7-4c99eb67b25a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:23.572193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0630b-e5ee-4fe1-9ec7-4c99eb67b25a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-d8a0630b-e5ee-4fe1-9ec7-4c99eb67b25a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:23.589868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:23.590778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:23.591682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:23.591682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:23.591682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:23.591682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:23.591682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 282/564] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:23.592190 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0630b-e5ee-4fe1-9ec7-4c99eb67b25a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:23.601781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d100f4-8a82-4313-99f7-f0fc917f75f7 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:23.636162 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee990bc-19e7-44ca-93c5-1a732ca03560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:23.636912 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee990bc-19e7-44ca-93c5-1a732ca03560 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.637216 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee990bc-19e7-44ca-93c5-1a732ca03560 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.637515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee990bc-19e7-44ca-93c5-1a732ca03560 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.638425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee990bc-19e7-44ca-93c5-1a732ca03560 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:23.638762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee990bc-19e7-44ca-93c5-1a732ca03560 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:23.640667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee990bc-19e7-44ca-93c5-1a732ca03560 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:23.657721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 283/565] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Jul 12 03:36:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:23.658946 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca4523-3aab-46a3-a334-abec01d2cc1f None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.659237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca4523-3aab-46a3-a334-abec01d2cc1f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.659511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca4523-3aab-46a3-a334-abec01d2cc1f None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.660872 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 284/566] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:23.661930 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa514b7-4dc2-4887-b91c-dc47c278e662 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.662228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa514b7-4dc2-4887-b91c-dc47c278e662 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.662484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa514b7-4dc2-4887-b91c-dc47c278e662 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.670008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 285/567] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:23.670008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa514b7-4dc2-4887-b91c-dc47c278e662 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:23.697873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0630b-e5ee-4fe1-9ec7-4c99eb67b25a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:23.711027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:23.732460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376c368-1f8e-4672-b30f-94975724fd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:23.733257 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376c368-1f8e-4672-b30f-94975724fd68 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.733538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376c368-1f8e-4672-b30f-94975724fd68 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.733817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376c368-1f8e-4672-b30f-94975724fd68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.740787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376c368-1f8e-4672-b30f-94975724fd68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:23.742301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376c368-1f8e-4672-b30f-94975724fd68 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-8376c368-1f8e-4672-b30f-94975724fd68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:23.756937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:23.757801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:23.758668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:23.758668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:23.758668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:23.758668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:23.758668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:23.758668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:23.758668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:23.758668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:23.758668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 283/568] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:23.759176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376c368-1f8e-4672-b30f-94975724fd68 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:23.770397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:23.777722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa514b7-4dc2-4887-b91c-dc47c278e662 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:23.811930 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e53d98-b3aa-437e-9ba2-280a18ff9b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:23.812629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e53d98-b3aa-437e-9ba2-280a18ff9b26 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.812910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e53d98-b3aa-437e-9ba2-280a18ff9b26 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.813187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e53d98-b3aa-437e-9ba2-280a18ff9b26 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.813906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e53d98-b3aa-437e-9ba2-280a18ff9b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:23.814212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e53d98-b3aa-437e-9ba2-280a18ff9b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:23.815599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e53d98-b3aa-437e-9ba2-280a18ff9b26 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:23.826766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 286/569] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:23.827784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba3bbd-e8b8-48b4-8fe6-4efdf59cfb5e None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.828067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba3bbd-e8b8-48b4-8fe6-4efdf59cfb5e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.828356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba3bbd-e8b8-48b4-8fe6-4efdf59cfb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.844519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-9dba3bbd-e8b8-48b4-8fe6-4efdf59cfb5e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:23.845657 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-9dba3bbd-e8b8-48b4-8fe6-4efdf59cfb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:23.860072 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376c368-1f8e-4672-b30f-94975724fd68 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:23.893804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dac02-0b45-4595-996b-ef1c505f0373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:23.894658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dac02-0b45-4595-996b-ef1c505f0373 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.894937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dac02-0b45-4595-996b-ef1c505f0373 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.895208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dac02-0b45-4595-996b-ef1c505f0373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.904955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140dac02-0b45-4595-996b-ef1c505f0373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:23.906416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140dac02-0b45-4595-996b-ef1c505f0373 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:23.913424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 287/570] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:23.914713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a6bd0-c56f-4e2d-92c2-c84b797372ba None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:23.915001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a6bd0-c56f-4e2d-92c2-c84b797372ba None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:23.915278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a6bd0-c56f-4e2d-92c2-c84b797372ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-140dac02-0b45-4595-996b-ef1c505f0373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:23.927026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:23.928264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:23.929441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:23.929441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:23.929441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:23.929441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:23.929441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 284/571] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:23.930247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dac02-0b45-4595-996b-ef1c505f0373 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:23.936924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a13a6bd0-c56f-4e2d-92c2-c84b797372ba None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:23.938179 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a13a6bd0-c56f-4e2d-92c2-c84b797372ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:23.958238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:24.010629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 288/572] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:24.011479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a6bd0-c56f-4e2d-92c2-c84b797372ba None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:24.029263 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:24.039289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dac02-0b45-4595-996b-ef1c505f0373 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:24.072412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527041c5-8aef-46f7-b275-bf3d1c5c8a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:24.073083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527041c5-8aef-46f7-b275-bf3d1c5c8a97 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:24.073357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527041c5-8aef-46f7-b275-bf3d1c5c8a97 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:24.073641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527041c5-8aef-46f7-b275-bf3d1c5c8a97 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:24.074428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527041c5-8aef-46f7-b275-bf3d1c5c8a97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:24.074732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527041c5-8aef-46f7-b275-bf3d1c5c8a97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:24.076228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527041c5-8aef-46f7-b275-bf3d1c5c8a97 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:24.087988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 285/573] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:24.088810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527041c5-8aef-46f7-b275-bf3d1c5c8a97 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:24.110457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a6bd0-c56f-4e2d-92c2-c84b797372ba None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:24.143984 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c186c07-7538-495c-a89f-5f3a49dc5093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:24.144832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186c07-7538-495c-a89f-5f3a49dc5093 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:24.145119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186c07-7538-495c-a89f-5f3a49dc5093 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:24.145394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186c07-7538-495c-a89f-5f3a49dc5093 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:24.152362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c186c07-7538-495c-a89f-5f3a49dc5093 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:24.153872 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c186c07-7538-495c-a89f-5f3a49dc5093 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-4c186c07-7538-495c-a89f-5f3a49dc5093 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:24.168050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:24.169099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:24.170204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:24.170204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:24.170204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:24.170204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:24.170204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:24.170204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:24.170204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:24.170204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:24.170204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:36:24.170204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:24.172145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 289/574] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 12 03:36:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:24.173108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c186c07-7538-495c-a89f-5f3a49dc5093 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:24.174163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:24.190905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527041c5-8aef-46f7-b275-bf3d1c5c8a97 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:24.222815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04343c-0c99-4063-92a3-59608668d1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:24.223471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04343c-0c99-4063-92a3-59608668d1e3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:24.223753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04343c-0c99-4063-92a3-59608668d1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:24.224050 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04343c-0c99-4063-92a3-59608668d1e3 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:24.224770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04343c-0c99-4063-92a3-59608668d1e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:24.225075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04343c-0c99-4063-92a3-59608668d1e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:24.226468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04343c-0c99-4063-92a3-59608668d1e3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:24.237887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 286/575] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:24.238662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04343c-0c99-4063-92a3-59608668d1e3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:24.246963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:24.271364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c186c07-7538-495c-a89f-5f3a49dc5093 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:24.308152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3b163-ed81-46e3-9500-740ee8494b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:24.309241 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3b163-ed81-46e3-9500-740ee8494b62 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:24.309529 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3b163-ed81-46e3-9500-740ee8494b62 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:24.309802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3b163-ed81-46e3-9500-740ee8494b62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:24.320954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3b163-ed81-46e3-9500-740ee8494b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-2cd3b163-ed81-46e3-9500-740ee8494b62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:24.328654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:24.329585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:24.330402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 290/576] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:24.331079 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3b163-ed81-46e3-9500-740ee8494b62 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:24.341799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04343c-0c99-4063-92a3-59608668d1e3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:24.377044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7030-793c-4628-a020-c489f0ed17ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:24.377779 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7030-793c-4628-a020-c489f0ed17ea None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:24.378059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7030-793c-4628-a020-c489f0ed17ea None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:24.378325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7030-793c-4628-a020-c489f0ed17ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:24.388297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a7030-793c-4628-a020-c489f0ed17ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-206a7030-793c-4628-a020-c489f0ed17ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:24.396649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:24.397583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:24.398412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 287/577] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:24.399098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7030-793c-4628-a020-c489f0ed17ea None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:24.404505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:24.435337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3b163-ed81-46e3-9500-740ee8494b62 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:24.468121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc912c-c767-475b-81bc-d07748996b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:24.468770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc912c-c767-475b-81bc-d07748996b17 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:24.469059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc912c-c767-475b-81bc-d07748996b17 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:24.469337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc912c-c767-475b-81bc-d07748996b17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:24.470187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc912c-c767-475b-81bc-d07748996b17 None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=0026b8f11158492b90da3395cc2c43c6) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:24.470494 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc912c-c767-475b-81bc-d07748996b17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:24.471985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc912c-c767-475b-81bc-d07748996b17 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:24.474837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:24.492512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 291/578] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:24 2022] GET /identity/v3/role_assignments?user.id=0026b8f11158492b90da3395cc2c43c6&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:24.493595 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc912c-c767-475b-81bc-d07748996b17 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:24.504221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7030-793c-4628-a020-c489f0ed17ea None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:24.543606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232660c8-9eeb-4f03-abfe-4244a615be8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:24.544608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232660c8-9eeb-4f03-abfe-4244a615be8a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:24.544897 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232660c8-9eeb-4f03-abfe-4244a615be8a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:24.545191 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232660c8-9eeb-4f03-abfe-4244a615be8a None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:24.546070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232660c8-9eeb-4f03-abfe-4244a615be8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:24.546384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232660c8-9eeb-4f03-abfe-4244a615be8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:24.547986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232660c8-9eeb-4f03-abfe-4244a615be8a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:24.575095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 288/579] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 12 03:36:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:24.575095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232660c8-9eeb-4f03-abfe-4244a615be8a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:24.615034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc912c-c767-475b-81bc-d07748996b17 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:24.643197 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:24.655538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe880ad-1d22-4873-8a20-b168dc7155a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:24.655538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe880ad-1d22-4873-8a20-b168dc7155a5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:24.655538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe880ad-1d22-4873-8a20-b168dc7155a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:24.655538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe880ad-1d22-4873-8a20-b168dc7155a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:24.661451 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe880ad-1d22-4873-8a20-b168dc7155a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:24.662999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe880ad-1d22-4873-8a20-b168dc7155a5 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-0fe880ad-1d22-4873-8a20-b168dc7155a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:24.677580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:24.678480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:24.679362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:24.679362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:24.679362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:24.679362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:24.679362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:24.679362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:24.679362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:24.679362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:24.683596 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 292/580] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:24.684430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe880ad-1d22-4873-8a20-b168dc7155a5 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:24.692777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232660c8-9eeb-4f03-abfe-4244a615be8a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:24.728530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a883900-e03f-47f3-8cfc-26c0117a058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:24.729212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a883900-e03f-47f3-8cfc-26c0117a058b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:24.729466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a883900-e03f-47f3-8cfc-26c0117a058b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:24.729733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a883900-e03f-47f3-8cfc-26c0117a058b None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:24.730520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a883900-e03f-47f3-8cfc-26c0117a058b None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:24.730818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a883900-e03f-47f3-8cfc-26c0117a058b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:24.732233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a883900-e03f-47f3-8cfc-26c0117a058b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:24.740659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 289/581] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:24.741462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a883900-e03f-47f3-8cfc-26c0117a058b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:24.753075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:24.796068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe880ad-1d22-4873-8a20-b168dc7155a5 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:24.833466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb00a5-6e43-44ee-af15-79a32aa2aa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:24.834154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb00a5-6e43-44ee-af15-79a32aa2aa51 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:24.834463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb00a5-6e43-44ee-af15-79a32aa2aa51 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:24.834674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb00a5-6e43-44ee-af15-79a32aa2aa51 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:24.835525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb00a5-6e43-44ee-af15-79a32aa2aa51 None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:24.835725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb00a5-6e43-44ee-af15-79a32aa2aa51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:24.837181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb00a5-6e43-44ee-af15-79a32aa2aa51 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:24.845560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 293/582] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:24.846171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb00a5-6e43-44ee-af15-79a32aa2aa51 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:24.860027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a883900-e03f-47f3-8cfc-26c0117a058b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:24.891474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:24.903444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e743783-4bb2-4dff-9474-39e9ea2017d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:24.904217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e743783-4bb2-4dff-9474-39e9ea2017d9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:24.904502 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e743783-4bb2-4dff-9474-39e9ea2017d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:24.904779 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e743783-4bb2-4dff-9474-39e9ea2017d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:24.916673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e743783-4bb2-4dff-9474-39e9ea2017d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:24.918156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e743783-4bb2-4dff-9474-39e9ea2017d9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-4e743783-4bb2-4dff-9474-39e9ea2017d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:24.937406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:24.938385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:24.939281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:24.939281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:24.939281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:24.939281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:24.939281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 290/583] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:24.939560 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e743783-4bb2-4dff-9474-39e9ea2017d9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:24.969750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb00a5-6e43-44ee-af15-79a32aa2aa51 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:24.993613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:25.006140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aecf78-3efe-4648-9877-66bbf030a7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:25.006807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aecf78-3efe-4648-9877-66bbf030a7f0 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:25.007095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aecf78-3efe-4648-9877-66bbf030a7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:25.007386 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aecf78-3efe-4648-9877-66bbf030a7f0 None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:25.008230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aecf78-3efe-4648-9877-66bbf030a7f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:25.008545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aecf78-3efe-4648-9877-66bbf030a7f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:25.010032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aecf78-3efe-4648-9877-66bbf030a7f0 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:25.027417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 294/584] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:25.027417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aecf78-3efe-4648-9877-66bbf030a7f0 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:25.053490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e743783-4bb2-4dff-9474-39e9ea2017d9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:25.089124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b701f9-6695-4f4f-964f-f601f173e74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:25.089925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b701f9-6695-4f4f-964f-f601f173e74d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:25.090210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b701f9-6695-4f4f-964f-f601f173e74d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:25.090484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b701f9-6695-4f4f-964f-f601f173e74d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:25.101160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b701f9-6695-4f4f-964f-f601f173e74d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:25.102624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b701f9-6695-4f4f-964f-f601f173e74d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-30b701f9-6695-4f4f-964f-f601f173e74d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:25.119973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:25.120941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:25.121824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:25.121824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:25.121824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:25.121824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:25.121824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 291/585] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:25.122104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b701f9-6695-4f4f-964f-f601f173e74d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:25.130675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aecf78-3efe-4648-9877-66bbf030a7f0 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:25.152273 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:25.169922 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e891a6-b906-4a3f-a0c9-4941603f326c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:25.170647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e891a6-b906-4a3f-a0c9-4941603f326c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:25.170935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e891a6-b906-4a3f-a0c9-4941603f326c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:25.171321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e891a6-b906-4a3f-a0c9-4941603f326c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:25.182637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e891a6-b906-4a3f-a0c9-4941603f326c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:25.184132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e891a6-b906-4a3f-a0c9-4941603f326c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-29e891a6-b906-4a3f-a0c9-4941603f326c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:25.202203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:25.203132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:25.203988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:25.203988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:25.203988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:25.203988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:25.208136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 295/586] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:25.208974 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e891a6-b906-4a3f-a0c9-4941603f326c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:25.229482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b701f9-6695-4f4f-964f-f601f173e74d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:25.242144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:25.265737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5adf5a-6010-4dd8-8090-35d1660110ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:25.266399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5adf5a-6010-4dd8-8090-35d1660110ac None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:25.266680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5adf5a-6010-4dd8-8090-35d1660110ac None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:25.266950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5adf5a-6010-4dd8-8090-35d1660110ac None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:25.267691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5adf5a-6010-4dd8-8090-35d1660110ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:25.268015 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5adf5a-6010-4dd8-8090-35d1660110ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:25.269466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5adf5a-6010-4dd8-8090-35d1660110ac None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:25.286322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 292/587] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:25.286322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5adf5a-6010-4dd8-8090-35d1660110ac None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:25.311977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e891a6-b906-4a3f-a0c9-4941603f326c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:25.345647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381757ad-e592-4966-9b6d-2ffe84ac3d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:25.346531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381757ad-e592-4966-9b6d-2ffe84ac3d74 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:25.346815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381757ad-e592-4966-9b6d-2ffe84ac3d74 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:25.347118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381757ad-e592-4966-9b6d-2ffe84ac3d74 None admin] PATH_INFO: `/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/7297224dc73c4358b60f1b096c0fb5bf/roles/f653b35aa347415e953be7ead75d9fd0` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:25.361176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381757ad-e592-4966-9b6d-2ffe84ac3d74 None admin] RBAC: Authorizing `identity:create_grant(project_id=48b17053daa84e4fa94840777cb1bee6, user_id=7297224dc73c4358b60f1b096c0fb5bf, role_id=f653b35aa347415e953be7ead75d9fd0)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:25.362691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381757ad-e592-4966-9b6d-2ffe84ac3d74 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:25.375024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:25.384040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 296/588] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:36:25 2022] PUT /identity/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/7297224dc73c4358b60f1b096c0fb5bf/roles/f653b35aa347415e953be7ead75d9fd0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:36:25.384040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381757ad-e592-4966-9b6d-2ffe84ac3d74 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:25.404815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5adf5a-6010-4dd8-8090-35d1660110ac None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:25.457476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058a370-a620-4c55-91e3-9f465e94544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:25.458171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a370-a620-4c55-91e3-9f465e94544c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:25.458521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a370-a620-4c55-91e3-9f465e94544c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:25.458860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a370-a620-4c55-91e3-9f465e94544c None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:25.459936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058a370-a620-4c55-91e3-9f465e94544c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:25.460333 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058a370-a620-4c55-91e3-9f465e94544c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:25.462492 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058a370-a620-4c55-91e3-9f465e94544c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:25.479263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 293/589] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:25.480304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058a370-a620-4c55-91e3-9f465e94544c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:25.490008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:25.511599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381757ad-e592-4966-9b6d-2ffe84ac3d74 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:25.555436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca8b80-c07b-466c-af77-2f92cd0bc2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:25.556030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca8b80-c07b-466c-af77-2f92cd0bc2e8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:25.556937 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca8b80-c07b-466c-af77-2f92cd0bc2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:25.557315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca8b80-c07b-466c-af77-2f92cd0bc2e8 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:25.559435 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca8b80-c07b-466c-af77-2f92cd0bc2e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:25.559997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca8b80-c07b-466c-af77-2f92cd0bc2e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:25.561598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca8b80-c07b-466c-af77-2f92cd0bc2e8 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:25.577510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 297/590] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:25.578376 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca8b80-c07b-466c-af77-2f92cd0bc2e8 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:25.626322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058a370-a620-4c55-91e3-9f465e94544c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:25.672264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4bfa7-c0b1-4295-90fb-cfb73de75fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:25.673130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4bfa7-c0b1-4295-90fb-cfb73de75fbb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:25.673413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4bfa7-c0b1-4295-90fb-cfb73de75fbb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:25.673742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4bfa7-c0b1-4295-90fb-cfb73de75fbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:25.685562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4bfa7-c0b1-4295-90fb-cfb73de75fbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:25.688175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4bfa7-c0b1-4295-90fb-cfb73de75fbb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:25.702244 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca8b80-c07b-466c-af77-2f92cd0bc2e8 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6bd4bfa7-c0b1-4295-90fb-cfb73de75fbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:25.708084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:25.708918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:25.709789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:25.709789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:25.709789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:25.709789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:25.709789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:25.709789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:25.709789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:25.709789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:25.710213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 294/591] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:25.710862 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4bfa7-c0b1-4295-90fb-cfb73de75fbb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:25.728431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:25.745668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53d93b-f716-4bcc-81b5-f5aa52a16234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:25.746815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53d93b-f716-4bcc-81b5-f5aa52a16234 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:25.747443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53d93b-f716-4bcc-81b5-f5aa52a16234 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:25.747795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53d93b-f716-4bcc-81b5-f5aa52a16234 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:25.759085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53d93b-f716-4bcc-81b5-f5aa52a16234 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:25.761130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53d93b-f716-4bcc-81b5-f5aa52a16234 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-5f53d93b-f716-4bcc-81b5-f5aa52a16234 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:25.777294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:25.778341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:25.779503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:25.779503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:25.779503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:25.779503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:25.779503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:25.779503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:25.779503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:25.779503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:25.779503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 03:36:25.779503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:25.784546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 298/592] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:25.785467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53d93b-f716-4bcc-81b5-f5aa52a16234 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:25.790363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:25.842585 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4bfa7-c0b1-4295-90fb-cfb73de75fbb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:25.886935 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72108d5a-5131-4f7b-a790-d3bccd82823c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:25.888486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72108d5a-5131-4f7b-a790-d3bccd82823c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:25.888486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72108d5a-5131-4f7b-a790-d3bccd82823c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:25.888711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72108d5a-5131-4f7b-a790-d3bccd82823c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:25.894370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53d93b-f716-4bcc-81b5-f5aa52a16234 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:25.897080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72108d5a-5131-4f7b-a790-d3bccd82823c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:25.900685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72108d5a-5131-4f7b-a790-d3bccd82823c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-72108d5a-5131-4f7b-a790-d3bccd82823c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:25.915204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:25.916425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:25.918193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:25.918193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:25.918193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:25.918193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:25.918193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:25.918193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:25.918193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:25.918193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:25.918193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 03:36:25.918193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:25.919443 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 295/593] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:25.919593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72108d5a-5131-4f7b-a790-d3bccd82823c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:25.941661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44a240-d5ec-4d2b-ac6f-1311be99cf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:25.942335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44a240-d5ec-4d2b-ac6f-1311be99cf80 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:25.948024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44a240-d5ec-4d2b-ac6f-1311be99cf80 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:25.948024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44a240-d5ec-4d2b-ac6f-1311be99cf80 None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:25.948024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44a240-d5ec-4d2b-ac6f-1311be99cf80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:25.948024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44a240-d5ec-4d2b-ac6f-1311be99cf80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:25.948024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44a240-d5ec-4d2b-ac6f-1311be99cf80 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:25.964329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 299/594] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:25.965791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44a240-d5ec-4d2b-ac6f-1311be99cf80 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:26.011507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:26.039956 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:26.062431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72108d5a-5131-4f7b-a790-d3bccd82823c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:26.080423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44a240-d5ec-4d2b-ac6f-1311be99cf80 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:26.108385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba9138-31dc-4d63-962e-f5276ddddab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:26.109481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba9138-31dc-4d63-962e-f5276ddddab5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:26.109775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba9138-31dc-4d63-962e-f5276ddddab5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:26.110061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba9138-31dc-4d63-962e-f5276ddddab5 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:26.111206 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba9138-31dc-4d63-962e-f5276ddddab5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:26.111516 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba9138-31dc-4d63-962e-f5276ddddab5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:26.113697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba9138-31dc-4d63-962e-f5276ddddab5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:26.118320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce984fab-030c-4b9e-925e-73c268cf6d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:26.119327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce984fab-030c-4b9e-925e-73c268cf6d00 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:26.119808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce984fab-030c-4b9e-925e-73c268cf6d00 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:26.120310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce984fab-030c-4b9e-925e-73c268cf6d00 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:26.121436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce984fab-030c-4b9e-925e-73c268cf6d00 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:26.121942 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce984fab-030c-4b9e-925e-73c268cf6d00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:26.123642 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce984fab-030c-4b9e-925e-73c268cf6d00 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:26.140036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 296/595] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Jul 12 03:36:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:26.140036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba9138-31dc-4d63-962e-f5276ddddab5 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:26.148631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 300/596] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Jul 12 03:36:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:26.149791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce984fab-030c-4b9e-925e-73c268cf6d00 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:26.270570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce984fab-030c-4b9e-925e-73c268cf6d00 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:26.275013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba9138-31dc-4d63-962e-f5276ddddab5 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:26.301275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:26.302467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:26.312676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cd3cc-d7f7-405c-9e8b-0254d05e446c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:26.313638 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cd3cc-d7f7-405c-9e8b-0254d05e446c None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:26.313965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cd3cc-d7f7-405c-9e8b-0254d05e446c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:26.314334 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901cd3cc-d7f7-405c-9e8b-0254d05e446c None admin] PATH_INFO: `/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/7138318e53ff4b7885942d49b19eb0a5/roles/f653b35aa347415e953be7ead75d9fd0` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:26.318115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912327b-9597-4b49-a3ef-b9c5e317255b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:26.319919 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912327b-9597-4b49-a3ef-b9c5e317255b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:26.320377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912327b-9597-4b49-a3ef-b9c5e317255b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:26.320798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912327b-9597-4b49-a3ef-b9c5e317255b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:26.330851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901cd3cc-d7f7-405c-9e8b-0254d05e446c None admin] RBAC: Authorizing `identity:create_grant(project_id=48b17053daa84e4fa94840777cb1bee6, user_id=7138318e53ff4b7885942d49b19eb0a5, role_id=f653b35aa347415e953be7ead75d9fd0)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:26.332605 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901cd3cc-d7f7-405c-9e8b-0254d05e446c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:26.334120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912327b-9597-4b49-a3ef-b9c5e317255b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:26.336462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912327b-9597-4b49-a3ef-b9c5e317255b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:26.350855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 297/597] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:36:26 2022] PUT /identity/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/7138318e53ff4b7885942d49b19eb0a5/roles/f653b35aa347415e953be7ead75d9fd0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:36:26.352029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cd3cc-d7f7-405c-9e8b-0254d05e446c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-a912327b-9597-4b49-a3ef-b9c5e317255b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:26.358237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:26.359400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:26.360562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:26.360562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:26.360562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:26.360562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:26.362242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 301/598] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:26.368110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912327b-9597-4b49-a3ef-b9c5e317255b None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:26.488134 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901cd3cc-d7f7-405c-9e8b-0254d05e446c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:26.493007 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912327b-9597-4b49-a3ef-b9c5e317255b None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:26.526176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6cdb4-3d7d-435e-8a1b-08a17786857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:26.527104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6cdb4-3d7d-435e-8a1b-08a17786857d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:26.527491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6cdb4-3d7d-435e-8a1b-08a17786857d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:26.527862 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e6cdb4-3d7d-435e-8a1b-08a17786857d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:26.539660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761da69e-06ba-4cf9-82e2-9005070dbd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:26.540448 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6cdb4-3d7d-435e-8a1b-08a17786857d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:26.540857 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761da69e-06ba-4cf9-82e2-9005070dbd4e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:26.541293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761da69e-06ba-4cf9-82e2-9005070dbd4e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:26.541751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761da69e-06ba-4cf9-82e2-9005070dbd4e None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:26.542456 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e6cdb4-3d7d-435e-8a1b-08a17786857d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:26.542851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761da69e-06ba-4cf9-82e2-9005070dbd4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:26.543369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761da69e-06ba-4cf9-82e2-9005070dbd4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:26.545231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761da69e-06ba-4cf9-82e2-9005070dbd4e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:26.562401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 302/599] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:26.564094 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161f6e1-8f48-4776-a38b-581379bf275d None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:26.564569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161f6e1-8f48-4776-a38b-581379bf275d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:26.566064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161f6e1-8f48-4776-a38b-581379bf275d None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-57e6cdb4-3d7d-435e-8a1b-08a17786857d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:26.567965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:26.568868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:26.569755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:26.569755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:26.569755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:26.569755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:26.569755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 298/600] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:26.570618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 303/601] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:26.571137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6cdb4-3d7d-435e-8a1b-08a17786857d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:26.576047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d718ba-1d7d-4df7-9258-db7b93c6efc7 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:26.576047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d718ba-1d7d-4df7-9258-db7b93c6efc7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:26.576047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d718ba-1d7d-4df7-9258-db7b93c6efc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:26.594759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-70d718ba-1d7d-4df7-9258-db7b93c6efc7 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:26.596346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-70d718ba-1d7d-4df7-9258-db7b93c6efc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:26.610944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:26.629214 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:26.680534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 304/602] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:26.681394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d718ba-1d7d-4df7-9258-db7b93c6efc7 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:26.722940 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e6cdb4-3d7d-435e-8a1b-08a17786857d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:26.760414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85e3a2-91f8-4e22-9217-d583d682c9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:26.761214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85e3a2-91f8-4e22-9217-d583d682c9c7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:26.761499 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85e3a2-91f8-4e22-9217-d583d682c9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:26.761828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85e3a2-91f8-4e22-9217-d583d682c9c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:26.769594 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85e3a2-91f8-4e22-9217-d583d682c9c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:26.771076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85e3a2-91f8-4e22-9217-d583d682c9c7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-8f85e3a2-91f8-4e22-9217-d583d682c9c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:26.786001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:26.786919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:26.787797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:26.787797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:26.787797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:26.787797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:26.787797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:26.787797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:26.787797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:26.787797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:26.787797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 299/603] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:26.792544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85e3a2-91f8-4e22-9217-d583d682c9c7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:26.803587 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d718ba-1d7d-4df7-9258-db7b93c6efc7 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:26.849281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a832e1-582d-4287-aa64-c89a9c0b4116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:26.850481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a832e1-582d-4287-aa64-c89a9c0b4116 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:26.851209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a832e1-582d-4287-aa64-c89a9c0b4116 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:26.851659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a832e1-582d-4287-aa64-c89a9c0b4116 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:26.853121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a832e1-582d-4287-aa64-c89a9c0b4116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:26.853606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a832e1-582d-4287-aa64-c89a9c0b4116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:26.855655 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a832e1-582d-4287-aa64-c89a9c0b4116 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:26.875445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 305/604] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:26.876643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a832e1-582d-4287-aa64-c89a9c0b4116 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:26.889317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:26.904158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85e3a2-91f8-4e22-9217-d583d682c9c7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:26.909698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:26.946448 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671b1b5-27b8-42e0-a810-d9dd3e9cba8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:26.947441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671b1b5-27b8-42e0-a810-d9dd3e9cba8c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:26.947850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671b1b5-27b8-42e0-a810-d9dd3e9cba8c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:26.948225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671b1b5-27b8-42e0-a810-d9dd3e9cba8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:26.962691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671b1b5-27b8-42e0-a810-d9dd3e9cba8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-7671b1b5-27b8-42e0-a810-d9dd3e9cba8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:26.970791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:26.972040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:26.972954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:26.972954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:26.972954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:26.972954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:26.972954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:26.972954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:26.972954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:26.972954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:26.972954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:26.972954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:26.972954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:26.972954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:26.973889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 300/605] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:26.974843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671b1b5-27b8-42e0-a810-d9dd3e9cba8c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:27.004997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a832e1-582d-4287-aa64-c89a9c0b4116 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:27.048103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2e002-e48a-467c-8960-6363e14c62fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:27.049077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2e002-e48a-467c-8960-6363e14c62fd None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:27.049544 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2e002-e48a-467c-8960-6363e14c62fd None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:27.050051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2e002-e48a-467c-8960-6363e14c62fd None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:27.051180 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2e002-e48a-467c-8960-6363e14c62fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:27.052202 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2e002-e48a-467c-8960-6363e14c62fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:27.054103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2e002-e48a-467c-8960-6363e14c62fd None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:27.070328 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 306/606] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:27.071406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2e002-e48a-467c-8960-6363e14c62fd None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:27.092588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671b1b5-27b8-42e0-a810-d9dd3e9cba8c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:27.130361 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294794c9-0c58-43bc-b183-207a7e3c3bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:27.131714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294794c9-0c58-43bc-b183-207a7e3c3bde None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:27.132465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294794c9-0c58-43bc-b183-207a7e3c3bde None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:27.132921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294794c9-0c58-43bc-b183-207a7e3c3bde None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:27.142622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294794c9-0c58-43bc-b183-207a7e3c3bde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:27.144827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294794c9-0c58-43bc-b183-207a7e3c3bde None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-294794c9-0c58-43bc-b183-207a7e3c3bde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:27.163984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:27.164969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:27.165857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:27.165857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:27.165857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:27.165857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:27.165857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:27.165857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:27.165857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:27.165857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:27.173329 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 301/607] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:27.173329 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294794c9-0c58-43bc-b183-207a7e3c3bde None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:27.175660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:27.185936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:27.197908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2e002-e48a-467c-8960-6363e14c62fd None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:27.235441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166660ec-d19f-4b7a-89e6-04f8940194cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:27.236407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166660ec-d19f-4b7a-89e6-04f8940194cc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:27.236860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166660ec-d19f-4b7a-89e6-04f8940194cc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:27.237322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166660ec-d19f-4b7a-89e6-04f8940194cc None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:27.238309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166660ec-d19f-4b7a-89e6-04f8940194cc None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:27.238845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166660ec-d19f-4b7a-89e6-04f8940194cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:27.240564 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166660ec-d19f-4b7a-89e6-04f8940194cc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:27.250090 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 307/608] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:27.251257 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166660ec-d19f-4b7a-89e6-04f8940194cc None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:27.296439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294794c9-0c58-43bc-b183-207a7e3c3bde None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:27.340130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1757a7-9de2-4a6d-a713-336be2e8124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:27.341403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1757a7-9de2-4a6d-a713-336be2e8124d None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:27.341885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1757a7-9de2-4a6d-a713-336be2e8124d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:27.342441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1757a7-9de2-4a6d-a713-336be2e8124d None admin] PATH_INFO: `/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/139be21e7f644b5796d30158851ebd00/roles/f653b35aa347415e953be7ead75d9fd0` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:27.359205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1757a7-9de2-4a6d-a713-336be2e8124d None admin] RBAC: Authorizing `identity:create_grant(project_id=48b17053daa84e4fa94840777cb1bee6, user_id=139be21e7f644b5796d30158851ebd00, role_id=f653b35aa347415e953be7ead75d9fd0)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:27.361098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1757a7-9de2-4a6d-a713-336be2e8124d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:27.371755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166660ec-d19f-4b7a-89e6-04f8940194cc None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:27.382217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 302/609] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:36:27 2022] PUT /identity/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/139be21e7f644b5796d30158851ebd00/roles/f653b35aa347415e953be7ead75d9fd0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:36:27.383925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1757a7-9de2-4a6d-a713-336be2e8124d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:27.409729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a009b4-45d4-4da4-a428-f622093f888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:27.410425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a009b4-45d4-4da4-a428-f622093f888e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:27.410706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a009b4-45d4-4da4-a428-f622093f888e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:27.410984 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a009b4-45d4-4da4-a428-f622093f888e None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:27.411812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a009b4-45d4-4da4-a428-f622093f888e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:27.412187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a009b4-45d4-4da4-a428-f622093f888e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:27.413681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a009b4-45d4-4da4-a428-f622093f888e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:27.427925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 308/610] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:27.434218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a009b4-45d4-4da4-a428-f622093f888e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:27.444236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:27.476859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:27.535807 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1757a7-9de2-4a6d-a713-336be2e8124d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:27.591978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff298b-c933-446f-b252-ff3d35c2732e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:27.592765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff298b-c933-446f-b252-ff3d35c2732e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:27.593054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff298b-c933-446f-b252-ff3d35c2732e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:27.593408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff298b-c933-446f-b252-ff3d35c2732e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:27.607784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff298b-c933-446f-b252-ff3d35c2732e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:27.609351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff298b-c933-446f-b252-ff3d35c2732e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-02ff298b-c933-446f-b252-ff3d35c2732e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:27.630187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:27.631117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:27.632179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:27.632179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:27.632179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:27.632179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:27.632398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a009b4-45d4-4da4-a428-f622093f888e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:27.632487 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 303/611] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:27.639886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff298b-c933-446f-b252-ff3d35c2732e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:27.682033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b78cc-4120-4f3d-862f-c40aaf102675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:27.682939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b78cc-4120-4f3d-862f-c40aaf102675 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:27.683225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b78cc-4120-4f3d-862f-c40aaf102675 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:27.683503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b78cc-4120-4f3d-862f-c40aaf102675 None admin] PATH_INFO: `/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/cee6da907b614dc5b6ffb47e2860f459/roles/f653b35aa347415e953be7ead75d9fd0` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:27.703303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b78cc-4120-4f3d-862f-c40aaf102675 None admin] RBAC: Authorizing `identity:create_grant(project_id=48b17053daa84e4fa94840777cb1bee6, user_id=cee6da907b614dc5b6ffb47e2860f459, role_id=f653b35aa347415e953be7ead75d9fd0)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:27.705329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b78cc-4120-4f3d-862f-c40aaf102675 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:27.755933 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff298b-c933-446f-b252-ff3d35c2732e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:27.762556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 309/612] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:36:27 2022] PUT /identity/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/cee6da907b614dc5b6ffb47e2860f459/roles/f653b35aa347415e953be7ead75d9fd0 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:36:27.768888 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:27.791430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0168db-6e49-4b2d-bf8e-1a0dfaa7d293 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:27.791768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0168db-6e49-4b2d-bf8e-1a0dfaa7d293 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:27.792017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0168db-6e49-4b2d-bf8e-1a0dfaa7d293 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:27.793813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 310/613] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:27.796702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead10d65-706c-4b0f-8d2f-8333a77b9684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:27.797408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead10d65-706c-4b0f-8d2f-8333a77b9684 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:27.797905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead10d65-706c-4b0f-8d2f-8333a77b9684 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:27.798265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead10d65-706c-4b0f-8d2f-8333a77b9684 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:27.799017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead10d65-706c-4b0f-8d2f-8333a77b9684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:27.799340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead10d65-706c-4b0f-8d2f-8333a77b9684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:27.800247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721abb90-3e76-4172-9c09-d2a813439e11 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:27.800727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721abb90-3e76-4172-9c09-d2a813439e11 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:27.800954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead10d65-706c-4b0f-8d2f-8333a77b9684 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:27.801360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721abb90-3e76-4172-9c09-d2a813439e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:27.820331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-721abb90-3e76-4172-9c09-d2a813439e11 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:27.821892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-721abb90-3e76-4172-9c09-d2a813439e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:27.822983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 304/614] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:27.824418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead10d65-706c-4b0f-8d2f-8333a77b9684 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:27.916487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:27.945576 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 311/615] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:27.994665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721abb90-3e76-4172-9c09-d2a813439e11 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:28.014957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead10d65-706c-4b0f-8d2f-8333a77b9684 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:28.063590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a8a99-445d-421f-8900-bbeb6b79a57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:28.064352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a8a99-445d-421f-8900-bbeb6b79a57a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:28.064640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a8a99-445d-421f-8900-bbeb6b79a57a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:28.064919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a8a99-445d-421f-8900-bbeb6b79a57a None admin] PATH_INFO: `/v3/users/swift` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:28.078623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a8a99-445d-421f-8900-bbeb6b79a57a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:28.088155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a8a99-445d-421f-8900-bbeb6b79a57a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f45a8a99-445d-421f-8900-bbeb6b79a57a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:28.110989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:28.112122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:28.113054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:28.113054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:28.113054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:28.113054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:28.113054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:28.113054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:28.113054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:28.113054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:28.113054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 03:36:28.113054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:28.120956 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 305/616] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:28.124910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721abb90-3e76-4172-9c09-d2a813439e11 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:28.126185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a8a99-445d-421f-8900-bbeb6b79a57a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:28.165131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7e413-5c46-4988-8092-b15eb6074524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:28.167371 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7e413-5c46-4988-8092-b15eb6074524 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:28.168895 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7e413-5c46-4988-8092-b15eb6074524 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:28.169353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7e413-5c46-4988-8092-b15eb6074524 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:28.191110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7e413-5c46-4988-8092-b15eb6074524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:28.195313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-dfe7e413-5c46-4988-8092-b15eb6074524 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:28.206540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:28.207540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:28.208409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:28.208409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:28.208409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:28.208409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:28.208409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:28.208409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:28.208409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:28.208409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:28.208409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:28.208409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:28.208409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:28.208409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:28.217776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 312/617] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:28.224053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7e413-5c46-4988-8092-b15eb6074524 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:28.303326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:28.335624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a8a99-445d-421f-8900-bbeb6b79a57a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:28.389570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7e413-5c46-4988-8092-b15eb6074524 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:28.403660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afbbb6-223f-495e-91e5-cd5ba582f868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:28.404400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afbbb6-223f-495e-91e5-cd5ba582f868 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:28.404683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afbbb6-223f-495e-91e5-cd5ba582f868 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:28.404963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afbbb6-223f-495e-91e5-cd5ba582f868 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:28.405814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afbbb6-223f-495e-91e5-cd5ba582f868 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:28.406129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afbbb6-223f-495e-91e5-cd5ba582f868 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:28.407717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afbbb6-223f-495e-91e5-cd5ba582f868 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:28.436494 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b5742-62ff-4b1e-ad41-74badb87ed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:28.437493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5742-62ff-4b1e-ad41-74badb87ed25 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:28.437918 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5742-62ff-4b1e-ad41-74badb87ed25 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:28.438343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5742-62ff-4b1e-ad41-74badb87ed25 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:28.439232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5742-62ff-4b1e-ad41-74badb87ed25 None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:28.440890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5742-62ff-4b1e-ad41-74badb87ed25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:28.442471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5742-62ff-4b1e-ad41-74badb87ed25 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:28.447120 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 306/618] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Jul 12 03:36:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:28.452411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afbbb6-223f-495e-91e5-cd5ba582f868 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:28.466365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 313/619] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:28.467559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b5742-62ff-4b1e-ad41-74badb87ed25 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:28.608963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:28.623809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afbbb6-223f-495e-91e5-cd5ba582f868 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:28.657304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b5742-62ff-4b1e-ad41-74badb87ed25 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:28.669253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:28.675767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f25577-dda0-45d9-b573-79f2e2ef5f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:28.677470 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f25577-dda0-45d9-b573-79f2e2ef5f4e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:28.677761 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f25577-dda0-45d9-b573-79f2e2ef5f4e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:28.678040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f25577-dda0-45d9-b573-79f2e2ef5f4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:28.693528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f25577-dda0-45d9-b573-79f2e2ef5f4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:28.695912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f25577-dda0-45d9-b573-79f2e2ef5f4e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:28.699751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfac6d-4817-41fb-9656-60ece349c8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:28.700755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfac6d-4817-41fb-9656-60ece349c8d3 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:28.701163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfac6d-4817-41fb-9656-60ece349c8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:28.701538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfac6d-4817-41fb-9656-60ece349c8d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:28.714202 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfac6d-4817-41fb-9656-60ece349c8d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:28.715802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfac6d-4817-41fb-9656-60ece349c8d3 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-09f25577-dda0-45d9-b573-79f2e2ef5f4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:28.718845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:28.719727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:28.720849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:28.720849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:28.720849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:28.720849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:28.720849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 307/620] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:28.721747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173fa37-e06f-4ecb-b54b-a6328571a466 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:28.722036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173fa37-e06f-4ecb-b54b-a6328571a466 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:28.722315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173fa37-e06f-4ecb-b54b-a6328571a466 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:28.728350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 308/621] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:28.729180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173fa37-e06f-4ecb-b54b-a6328571a466 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-e0dfac6d-4817-41fb-9656-60ece349c8d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:28.741513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:28.742603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:28.743475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:28.743475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:28.743475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:28.743475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:28.746279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 314/622] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:28.748340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45262fa2-0447-4488-aba8-03d32efb54f7 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:28.748809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45262fa2-0447-4488-aba8-03d32efb54f7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:28.749489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45262fa2-0447-4488-aba8-03d32efb54f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:28.771261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-45262fa2-0447-4488-aba8-03d32efb54f7 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:28.773656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-45262fa2-0447-4488-aba8-03d32efb54f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:28.849326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173fa37-e06f-4ecb-b54b-a6328571a466 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:28.856116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 315/623] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:28.857718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45262fa2-0447-4488-aba8-03d32efb54f7 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:28.889922 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401762f1-0b6e-4faf-a6dc-2b374e0fc838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:28.890972 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401762f1-0b6e-4faf-a6dc-2b374e0fc838 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:28.891439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401762f1-0b6e-4faf-a6dc-2b374e0fc838 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:28.892059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401762f1-0b6e-4faf-a6dc-2b374e0fc838 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:28.893214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401762f1-0b6e-4faf-a6dc-2b374e0fc838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:28.893704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401762f1-0b6e-4faf-a6dc-2b374e0fc838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:28.895306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401762f1-0b6e-4faf-a6dc-2b374e0fc838 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:28.904872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:28.914362 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 309/624] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:28.915789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401762f1-0b6e-4faf-a6dc-2b374e0fc838 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:28.974695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:28.986375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45262fa2-0447-4488-aba8-03d32efb54f7 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:29.022906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f6bd8-b068-4d61-8d8d-a066e10bf1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:29.023593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401762f1-0b6e-4faf-a6dc-2b374e0fc838 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:29.024402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f6bd8-b068-4d61-8d8d-a066e10bf1bd None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.024843 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f6bd8-b068-4d61-8d8d-a066e10bf1bd None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.025372 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f6bd8-b068-4d61-8d8d-a066e10bf1bd None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.026341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f6bd8-b068-4d61-8d8d-a066e10bf1bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:29.026868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f6bd8-b068-4d61-8d8d-a066e10bf1bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:29.028548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f6bd8-b068-4d61-8d8d-a066e10bf1bd None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:29.041379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 316/625] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:29.042562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f6bd8-b068-4d61-8d8d-a066e10bf1bd None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:29.060153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179a432-51d0-4cec-b9c4-5f313f08652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:29.061010 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179a432-51d0-4cec-b9c4-5f313f08652a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.061344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179a432-51d0-4cec-b9c4-5f313f08652a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.061621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179a432-51d0-4cec-b9c4-5f313f08652a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.073029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179a432-51d0-4cec-b9c4-5f313f08652a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f179a432-51d0-4cec-b9c4-5f313f08652a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:29.081597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:29.082597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:29.083484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:29.083484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:29.083484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:29.083484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:29.083484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:29.083484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:29.083484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:29.083484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:29.083484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:29.083484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:29.083484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:29.083484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:29.087756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 310/626] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:29.088645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179a432-51d0-4cec-b9c4-5f313f08652a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:29.155340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:29.156641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f6bd8-b068-4d61-8d8d-a066e10bf1bd None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:29.194260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f595665-42af-4565-9500-9487a24d9c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:29.195433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f595665-42af-4565-9500-9487a24d9c5d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.196066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f595665-42af-4565-9500-9487a24d9c5d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.196566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f595665-42af-4565-9500-9487a24d9c5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.200301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179a432-51d0-4cec-b9c4-5f313f08652a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:29.204476 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f595665-42af-4565-9500-9487a24d9c5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:29.206247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f595665-42af-4565-9500-9487a24d9c5d None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-4f595665-42af-4565-9500-9487a24d9c5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:29.223509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:29.224495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:29.225390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:29.225390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:29.225390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:29.225390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:29.225390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:29.225390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:29.225390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:29.225390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:29.228029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 317/627] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:29.229032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f595665-42af-4565-9500-9487a24d9c5d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:29.243118 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe0a08-2ed8-47e1-aa99-9a5196794eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:29.244085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe0a08-2ed8-47e1-aa99-9a5196794eaf None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.244457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe0a08-2ed8-47e1-aa99-9a5196794eaf None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.244766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe0a08-2ed8-47e1-aa99-9a5196794eaf None admin] PATH_INFO: `/v3/users/glance` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.246093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:29.254662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe0a08-2ed8-47e1-aa99-9a5196794eaf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:29.256451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe0a08-2ed8-47e1-aa99-9a5196794eaf None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-1bbe0a08-2ed8-47e1-aa99-9a5196794eaf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:29.272708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:29.273890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:29.275081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:29.275081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:29.275081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:29.275081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:29.275081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:29.275081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:29.275081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:29.275081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:29.275081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 03:36:29.275081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:29.277441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 311/628] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:29.278479 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe0a08-2ed8-47e1-aa99-9a5196794eaf None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:29.343178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f595665-42af-4565-9500-9487a24d9c5d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:29.384085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487dfeb-7887-4211-adc7-4994b2fe7ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:29.384766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487dfeb-7887-4211-adc7-4994b2fe7ce4 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.385052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487dfeb-7887-4211-adc7-4994b2fe7ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.385330 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487dfeb-7887-4211-adc7-4994b2fe7ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.386081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487dfeb-7887-4211-adc7-4994b2fe7ce4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:29.386398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487dfeb-7887-4211-adc7-4994b2fe7ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:29.387794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487dfeb-7887-4211-adc7-4994b2fe7ce4 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:29.397100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 318/629] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:29.398341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487dfeb-7887-4211-adc7-4994b2fe7ce4 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:29.403937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe0a08-2ed8-47e1-aa99-9a5196794eaf None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:29.426875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:29.443169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda94414-2960-4984-a900-391c0596f801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:29.444194 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda94414-2960-4984-a900-391c0596f801 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.444725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda94414-2960-4984-a900-391c0596f801 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.445304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda94414-2960-4984-a900-391c0596f801 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.446286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda94414-2960-4984-a900-391c0596f801 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:29.446915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda94414-2960-4984-a900-391c0596f801 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:29.448925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda94414-2960-4984-a900-391c0596f801 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:29.463592 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 312/630] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:29.464989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda94414-2960-4984-a900-391c0596f801 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:29.468646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:29.500487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487dfeb-7887-4211-adc7-4994b2fe7ce4 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:29.535711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fead8-4e70-4069-a80a-ee5a81f8e8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:29.536404 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fead8-4e70-4069-a80a-ee5a81f8e8d4 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.536689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fead8-4e70-4069-a80a-ee5a81f8e8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.536972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fead8-4e70-4069-a80a-ee5a81f8e8d4 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.537817 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fead8-4e70-4069-a80a-ee5a81f8e8d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:29.538130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fead8-4e70-4069-a80a-ee5a81f8e8d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:29.539639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fead8-4e70-4069-a80a-ee5a81f8e8d4 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:29.558643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 319/631] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Jul 12 03:36:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:29.559459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fead8-4e70-4069-a80a-ee5a81f8e8d4 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:29.580301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda94414-2960-4984-a900-391c0596f801 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:29.623514 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3c1f5-842b-4779-a387-1c8d955c6c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:29.624909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c1f5-842b-4779-a387-1c8d955c6c23 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.625286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c1f5-842b-4779-a387-1c8d955c6c23 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.625567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c1f5-842b-4779-a387-1c8d955c6c23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.636941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3c1f5-842b-4779-a387-1c8d955c6c23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:29.638427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3c1f5-842b-4779-a387-1c8d955c6c23 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6ff3c1f5-842b-4779-a387-1c8d955c6c23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:29.657496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:29.658447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:29.659432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:29.659432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:29.659432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:29.659432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:29.659432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 313/632] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:29.659731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3c1f5-842b-4779-a387-1c8d955c6c23 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:29.674434 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fead8-4e70-4069-a80a-ee5a81f8e8d4 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:29.692611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:29.705929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:29.713258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a22ef6-bf48-417b-abc7-c7330a2f8748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:29.714157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a22ef6-bf48-417b-abc7-c7330a2f8748 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.714444 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a22ef6-bf48-417b-abc7-c7330a2f8748 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.714722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a22ef6-bf48-417b-abc7-c7330a2f8748 None admin] PATH_INFO: `/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/0026b8f11158492b90da3395cc2c43c6/roles/f653b35aa347415e953be7ead75d9fd0` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.729665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a22ef6-bf48-417b-abc7-c7330a2f8748 None admin] RBAC: Authorizing `identity:create_grant(project_id=48b17053daa84e4fa94840777cb1bee6, user_id=0026b8f11158492b90da3395cc2c43c6, role_id=f653b35aa347415e953be7ead75d9fd0)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:29.731117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a22ef6-bf48-417b-abc7-c7330a2f8748 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:29.751685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 320/633] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:36:29 2022] PUT /identity/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/0026b8f11158492b90da3395cc2c43c6/roles/f653b35aa347415e953be7ead75d9fd0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:36:29.751685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5470b-5d77-4a19-b919-124c12f2ea64 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.751685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5470b-5d77-4a19-b919-124c12f2ea64 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.751685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5470b-5d77-4a19-b919-124c12f2ea64 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.751685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 321/634] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:29.751685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5470b-5d77-4a19-b919-124c12f2ea64 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:29.776836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3c1f5-842b-4779-a387-1c8d955c6c23 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:29.818366 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dd882-d568-416d-b6d1-2e08987fee8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:29.819133 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dd882-d568-416d-b6d1-2e08987fee8e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.819455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dd882-d568-416d-b6d1-2e08987fee8e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.819734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dd882-d568-416d-b6d1-2e08987fee8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.900599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5470b-5d77-4a19-b919-124c12f2ea64 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:29.901211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09dd882-d568-416d-b6d1-2e08987fee8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:29.902910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09dd882-d568-416d-b6d1-2e08987fee8e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:29.942976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:29.952820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0658eff-04dd-4291-8a44-fe01d879e25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:29.953409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0658eff-04dd-4291-8a44-fe01d879e25b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.953705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0658eff-04dd-4291-8a44-fe01d879e25b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.954665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0658eff-04dd-4291-8a44-fe01d879e25b None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.954665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0658eff-04dd-4291-8a44-fe01d879e25b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:29.954950 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0658eff-04dd-4291-8a44-fe01d879e25b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:29.959261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0658eff-04dd-4291-8a44-fe01d879e25b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:29.972758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 322/635] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:29.974073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99571bd-71ca-43b0-9ebd-854b04505a17 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:29.974361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99571bd-71ca-43b0-9ebd-854b04505a17 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:29.974633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99571bd-71ca-43b0-9ebd-854b04505a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:29.998606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f99571bd-71ca-43b0-9ebd-854b04505a17 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:30.000945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f99571bd-71ca-43b0-9ebd-854b04505a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:30.038226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:30.039898 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 314/636] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:30.041626 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dd882-d568-416d-b6d1-2e08987fee8e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:30.080029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 323/637] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:30.080029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99571bd-71ca-43b0-9ebd-854b04505a17 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:30.153616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dd882-d568-416d-b6d1-2e08987fee8e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:30.184588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:30.190086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99571bd-71ca-43b0-9ebd-854b04505a17 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:30.190236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f4efa-70df-404c-8302-2f3e7caa5238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:30.190987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f4efa-70df-404c-8302-2f3e7caa5238 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:30.191271 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f4efa-70df-404c-8302-2f3e7caa5238 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:30.191579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f4efa-70df-404c-8302-2f3e7caa5238 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:30.198760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f4efa-70df-404c-8302-2f3e7caa5238 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:30.200272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f4efa-70df-404c-8302-2f3e7caa5238 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-671f4efa-70df-404c-8302-2f3e7caa5238 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:30.215002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:30.215986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:30.216975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:30.216975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:30.216975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:30.216975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:30.216975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:30.216975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:30.216975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:30.216975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:30.216975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:36:30.216975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:30.216975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 315/638] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 12 03:36:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:30.218133 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de7e2-773f-40b4-afcc-82f023960a0b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:30.218133 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de7e2-773f-40b4-afcc-82f023960a0b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:30.218662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1de7e2-773f-40b4-afcc-82f023960a0b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:30.219732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 316/639] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:30.224308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1de7e2-773f-40b4-afcc-82f023960a0b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:30.232436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f654063-c539-4db4-896d-03f87887412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:30.233248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f654063-c539-4db4-896d-03f87887412a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:30.233523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f654063-c539-4db4-896d-03f87887412a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:30.233803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f654063-c539-4db4-896d-03f87887412a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:30.244314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f654063-c539-4db4-896d-03f87887412a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:30.245775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f654063-c539-4db4-896d-03f87887412a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-1f654063-c539-4db4-896d-03f87887412a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:30.264633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:30.265554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:30.266447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:30.266447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:30.266447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:30.266447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:30.266447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 324/640] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:30.267191 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f654063-c539-4db4-896d-03f87887412a None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:30.293983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:30.334546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1de7e2-773f-40b4-afcc-82f023960a0b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:30.370320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702faff7-e89f-49f9-9fce-79d47cab5301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:30.371419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702faff7-e89f-49f9-9fce-79d47cab5301 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:30.371783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702faff7-e89f-49f9-9fce-79d47cab5301 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:30.371947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702faff7-e89f-49f9-9fce-79d47cab5301 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:30.382841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702faff7-e89f-49f9-9fce-79d47cab5301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:30.384862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f654063-c539-4db4-896d-03f87887412a None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-702faff7-e89f-49f9-9fce-79d47cab5301 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:30.390849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:30.392026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:30.392951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 317/641] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:30.393637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b585224-207c-4a8f-b90d-3abd0d9509d5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:30.394043 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b585224-207c-4a8f-b90d-3abd0d9509d5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:30.394202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b585224-207c-4a8f-b90d-3abd0d9509d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:30.410636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6b585224-207c-4a8f-b90d-3abd0d9509d5 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:30.411847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6b585224-207c-4a8f-b90d-3abd0d9509d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:30.429653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edb90f-70a8-4125-ad2f-69b133add805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:30.430331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edb90f-70a8-4125-ad2f-69b133add805 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:30.430734 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edb90f-70a8-4125-ad2f-69b133add805 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:30.430940 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78edb90f-70a8-4125-ad2f-69b133add805 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:30.431723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edb90f-70a8-4125-ad2f-69b133add805 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:30.432064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edb90f-70a8-4125-ad2f-69b133add805 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:30.433538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78edb90f-70a8-4125-ad2f-69b133add805 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:30.451166 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 325/642] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 12 03:36:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:30.452315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edb90f-70a8-4125-ad2f-69b133add805 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:30.465125 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:30.487166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 318/643] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:30.488259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b585224-207c-4a8f-b90d-3abd0d9509d5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:30.546960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:30.568415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78edb90f-70a8-4125-ad2f-69b133add805 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:30.601577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b585224-207c-4a8f-b90d-3abd0d9509d5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:30.602470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97114cf8-4676-4299-af83-4f605f0e0c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:30.603146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97114cf8-4676-4299-af83-4f605f0e0c72 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:30.603430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97114cf8-4676-4299-af83-4f605f0e0c72 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:30.603708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97114cf8-4676-4299-af83-4f605f0e0c72 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:30.604459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97114cf8-4676-4299-af83-4f605f0e0c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:30.604764 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97114cf8-4676-4299-af83-4f605f0e0c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:30.606147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97114cf8-4676-4299-af83-4f605f0e0c72 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:30.617960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 326/644] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:30.622968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97114cf8-4676-4299-af83-4f605f0e0c72 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:30.639296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971d91a-f842-44cd-b549-28aab4579c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:30.639981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971d91a-f842-44cd-b549-28aab4579c74 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:30.640293 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971d91a-f842-44cd-b549-28aab4579c74 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:30.640569 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971d91a-f842-44cd-b549-28aab4579c74 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:30.641318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971d91a-f842-44cd-b549-28aab4579c74 None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:30.641623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971d91a-f842-44cd-b549-28aab4579c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:30.643084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971d91a-f842-44cd-b549-28aab4579c74 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:30.651763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 319/645] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:30.652525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971d91a-f842-44cd-b549-28aab4579c74 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:30.699473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:30.729060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97114cf8-4676-4299-af83-4f605f0e0c72 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:30.755770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971d91a-f842-44cd-b549-28aab4579c74 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:30.761888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879ff81-055f-444a-b7e2-ae3997983ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:30.762550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879ff81-055f-444a-b7e2-ae3997983ec7 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:30.762838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879ff81-055f-444a-b7e2-ae3997983ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:30.763142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879ff81-055f-444a-b7e2-ae3997983ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:30.774052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:30.793331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9f43b-7bc6-48fc-b40a-ff803a3ce43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:30.794408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9f43b-7bc6-48fc-b40a-ff803a3ce43b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:30.794974 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9f43b-7bc6-48fc-b40a-ff803a3ce43b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:30.795316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9f43b-7bc6-48fc-b40a-ff803a3ce43b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:30.806055 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9f43b-7bc6-48fc-b40a-ff803a3ce43b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-48f9f43b-7bc6-48fc-b40a-ff803a3ce43b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:30.814288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:30.815384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 320/646] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:30.817026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9f43b-7bc6-48fc-b40a-ff803a3ce43b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:30.841504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879ff81-055f-444a-b7e2-ae3997983ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:30.844761 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879ff81-055f-444a-b7e2-ae3997983ec7 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:30.926173 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9f43b-7bc6-48fc-b40a-ff803a3ce43b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:30.928641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:30.962493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef5f8b-12c2-4c10-9cb6-001bc7cc5498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:30.963310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef5f8b-12c2-4c10-9cb6-001bc7cc5498 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:30.963595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef5f8b-12c2-4c10-9cb6-001bc7cc5498 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:30.963873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef5f8b-12c2-4c10-9cb6-001bc7cc5498 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:30.969854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 327/647] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:30.969854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879ff81-055f-444a-b7e2-ae3997983ec7 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:30.975319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef5f8b-12c2-4c10-9cb6-001bc7cc5498 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:30.976843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef5f8b-12c2-4c10-9cb6-001bc7cc5498 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-64ef5f8b-12c2-4c10-9cb6-001bc7cc5498 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:30.991668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:30.992696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:30.993606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:30.993606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:30.993606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:30.993606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:30.993606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:30.993606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:30.993606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:30.993606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:30.993606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 321/648] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:30.994127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef5f8b-12c2-4c10-9cb6-001bc7cc5498 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:31.012973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:31.078500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879ff81-055f-444a-b7e2-ae3997983ec7 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:31.111520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef5f8b-12c2-4c10-9cb6-001bc7cc5498 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:31.119418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad474ae7-07fb-4de3-98d4-1d3c8438b027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:31.120647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad474ae7-07fb-4de3-98d4-1d3c8438b027 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:31.120934 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad474ae7-07fb-4de3-98d4-1d3c8438b027 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:31.121208 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad474ae7-07fb-4de3-98d4-1d3c8438b027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:31.132240 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad474ae7-07fb-4de3-98d4-1d3c8438b027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:31.133777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad474ae7-07fb-4de3-98d4-1d3c8438b027 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:31.148470 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339bb15-84af-4b4a-824d-ba1414e19890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:31.149205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339bb15-84af-4b4a-824d-ba1414e19890 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:31.149522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339bb15-84af-4b4a-824d-ba1414e19890 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:31.149817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339bb15-84af-4b4a-824d-ba1414e19890 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:31.150586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339bb15-84af-4b4a-824d-ba1414e19890 None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:31.150891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339bb15-84af-4b4a-824d-ba1414e19890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:31.152297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339bb15-84af-4b4a-824d-ba1414e19890 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-ad474ae7-07fb-4de3-98d4-1d3c8438b027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:31.152931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:31.153820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:31.154742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:31.154742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:31.154742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:31.154742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:31.154742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 328/649] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:31.155029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad474ae7-07fb-4de3-98d4-1d3c8438b027 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:31.166207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 322/650] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:31.166978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339bb15-84af-4b4a-824d-ba1414e19890 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:31.197230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:31.258119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:31.268618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad474ae7-07fb-4de3-98d4-1d3c8438b027 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:31.275025 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339bb15-84af-4b4a-824d-ba1414e19890 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:31.305933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d264363-df02-414a-95bb-30e3eb3e6c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:31.307708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d264363-df02-414a-95bb-30e3eb3e6c81 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:31.308147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d264363-df02-414a-95bb-30e3eb3e6c81 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:31.308532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d264363-df02-414a-95bb-30e3eb3e6c81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:31.311383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8fa80-5fdd-4ad1-a3f8-91f216567c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:31.312083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8fa80-5fdd-4ad1-a3f8-91f216567c68 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:31.312416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8fa80-5fdd-4ad1-a3f8-91f216567c68 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:31.312692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8fa80-5fdd-4ad1-a3f8-91f216567c68 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:31.313520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8fa80-5fdd-4ad1-a3f8-91f216567c68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:31.313833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8fa80-5fdd-4ad1-a3f8-91f216567c68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:31.315332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8fa80-5fdd-4ad1-a3f8-91f216567c68 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:31.320332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d264363-df02-414a-95bb-30e3eb3e6c81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:31.321798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d264363-df02-414a-95bb-30e3eb3e6c81 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:31.334653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 323/651] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:31.334653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8fa80-5fdd-4ad1-a3f8-91f216567c68 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-7d264363-df02-414a-95bb-30e3eb3e6c81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:31.343385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:31.344498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:31.345375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:31.345375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:31.345375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:31.345375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:31.345375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 329/652] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:31.350482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d264363-df02-414a-95bb-30e3eb3e6c81 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:31.426275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:31.443689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8fa80-5fdd-4ad1-a3f8-91f216567c68 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:31.467592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d264363-df02-414a-95bb-30e3eb3e6c81 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:31.479934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ec953-7658-4b3b-8c06-568911a0d881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:31.480639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ec953-7658-4b3b-8c06-568911a0d881 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:31.480922 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ec953-7658-4b3b-8c06-568911a0d881 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:31.481200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ec953-7658-4b3b-8c06-568911a0d881 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:31.481923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ec953-7658-4b3b-8c06-568911a0d881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:31.482253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ec953-7658-4b3b-8c06-568911a0d881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:31.483624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ec953-7658-4b3b-8c06-568911a0d881 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:31.499894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 324/653] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:31.499894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ec953-7658-4b3b-8c06-568911a0d881 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:31.508152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561fcda-8e03-4fc9-90a8-401c04606e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:31.508955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561fcda-8e03-4fc9-90a8-401c04606e46 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:31.509525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561fcda-8e03-4fc9-90a8-401c04606e46 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:31.509525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561fcda-8e03-4fc9-90a8-401c04606e46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:31.521299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561fcda-8e03-4fc9-90a8-401c04606e46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:31.522767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561fcda-8e03-4fc9-90a8-401c04606e46 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:31.525988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-4561fcda-8e03-4fc9-90a8-401c04606e46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:31.542412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:31.543361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:31.544978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:31.544978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:31.544978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:31.544978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:31.544978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 330/654] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:31.544978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561fcda-8e03-4fc9-90a8-401c04606e46 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:31.612629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ec953-7658-4b3b-8c06-568911a0d881 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:31.652910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e326a51-4308-410e-967f-781985bad883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:31.653542 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e326a51-4308-410e-967f-781985bad883 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:31.653827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e326a51-4308-410e-967f-781985bad883 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:31.654095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e326a51-4308-410e-967f-781985bad883 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:31.654921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e326a51-4308-410e-967f-781985bad883 None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=7297224dc73c4358b60f1b096c0fb5bf) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:31.655233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e326a51-4308-410e-967f-781985bad883 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:31.656167 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561fcda-8e03-4fc9-90a8-401c04606e46 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:31.656710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e326a51-4308-410e-967f-781985bad883 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:31.668041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 325/655] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:31 2022] GET /identity/v3/role_assignments?user.id=7297224dc73c4358b60f1b096c0fb5bf&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:31.668041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e326a51-4308-410e-967f-781985bad883 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:31.693958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:31.702407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61237f06-3c31-4f59-87ab-109d9d609f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:31.703078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61237f06-3c31-4f59-87ab-109d9d609f22 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:31.703475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61237f06-3c31-4f59-87ab-109d9d609f22 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:31.703745 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61237f06-3c31-4f59-87ab-109d9d609f22 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:31.704665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61237f06-3c31-4f59-87ab-109d9d609f22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:31.705002 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61237f06-3c31-4f59-87ab-109d9d609f22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:31.706533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61237f06-3c31-4f59-87ab-109d9d609f22 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:31.725283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 331/656] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:31.726305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61237f06-3c31-4f59-87ab-109d9d609f22 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:31.790487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:31.799799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e326a51-4308-410e-967f-781985bad883 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:31.840233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61237f06-3c31-4f59-87ab-109d9d609f22 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:31.846452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1a5e5-d1a0-4877-9e7b-31a7f74c202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:31.847594 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1a5e5-d1a0-4877-9e7b-31a7f74c202d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:31.848078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1a5e5-d1a0-4877-9e7b-31a7f74c202d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:31.848519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1a5e5-d1a0-4877-9e7b-31a7f74c202d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:31.859987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1a5e5-d1a0-4877-9e7b-31a7f74c202d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:31.861789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1a5e5-d1a0-4877-9e7b-31a7f74c202d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-4bb1a5e5-d1a0-4877-9e7b-31a7f74c202d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:31.878970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:31.879959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:31.881376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:31.881376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:31.881376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:31.881376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:31.881376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:31.881376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:31.881376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:31.881376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:31.881376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 03:36:31.881376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:31.881964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67dfb1-8d2c-4da5-88b4-ed759ad72000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:31.882059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 326/657] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:31.882142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67dfb1-8d2c-4da5-88b4-ed759ad72000 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:31.882142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67dfb1-8d2c-4da5-88b4-ed759ad72000 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:31.882142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67dfb1-8d2c-4da5-88b4-ed759ad72000 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:31.882320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1a5e5-d1a0-4877-9e7b-31a7f74c202d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:31.882393 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67dfb1-8d2c-4da5-88b4-ed759ad72000 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:31.884417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67dfb1-8d2c-4da5-88b4-ed759ad72000 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:31.885945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67dfb1-8d2c-4da5-88b4-ed759ad72000 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:31.899902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 332/658] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:31.901311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704877c-22c7-44c9-b5e6-a04ee7ac9bb3 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:31.901595 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704877c-22c7-44c9-b5e6-a04ee7ac9bb3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:31.901862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704877c-22c7-44c9-b5e6-a04ee7ac9bb3 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:31.903248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 333/659] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:31.907315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704877c-22c7-44c9-b5e6-a04ee7ac9bb3 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:31.965130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:32.009225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1a5e5-d1a0-4877-9e7b-31a7f74c202d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:32.024190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704877c-22c7-44c9-b5e6-a04ee7ac9bb3 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:32.043780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:32.049747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c98a-84cf-4c65-8092-4c9a4eccc3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:32.050851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c98a-84cf-4c65-8092-4c9a4eccc3e4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:32.051137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c98a-84cf-4c65-8092-4c9a4eccc3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:32.051573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c98a-84cf-4c65-8092-4c9a4eccc3e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:32.059100 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827c98a-84cf-4c65-8092-4c9a4eccc3e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:32.061767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827c98a-84cf-4c65-8092-4c9a4eccc3e4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:32.062753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99235e6-7279-45d8-a745-0353786235c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:32.063448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99235e6-7279-45d8-a745-0353786235c7 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:32.063714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99235e6-7279-45d8-a745-0353786235c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:32.063994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99235e6-7279-45d8-a745-0353786235c7 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:32.064901 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99235e6-7279-45d8-a745-0353786235c7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:32.065212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99235e6-7279-45d8-a745-0353786235c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:32.066692 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99235e6-7279-45d8-a745-0353786235c7 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-2827c98a-84cf-4c65-8092-4c9a4eccc3e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:32.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:32.077588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:32.083923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:32.083923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:32.083923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:32.083923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:32.083923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:32.083923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:32.083923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:32.083923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:32.083923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 327/660] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:32.083923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c98a-84cf-4c65-8092-4c9a4eccc3e4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:32.088545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 334/661] 10.222.0.20 () {56 vars in 1139 bytes} [Tue Jul 12 03:36:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:32.093236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595706a-00ba-4081-9344-3074f0c2554e None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:32.093236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595706a-00ba-4081-9344-3074f0c2554e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:32.093236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595706a-00ba-4081-9344-3074f0c2554e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:32.108870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2595706a-00ba-4081-9344-3074f0c2554e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:32.110129 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2595706a-00ba-4081-9344-3074f0c2554e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:32.182235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 335/662] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:32.183369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595706a-00ba-4081-9344-3074f0c2554e None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:32.196707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c98a-84cf-4c65-8092-4c9a4eccc3e4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:32.228188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:32.235008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a4bd9-51ce-4da9-b1e2-9b474f8a877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:32.235576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a4bd9-51ce-4da9-b1e2-9b474f8a877c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:32.236213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a4bd9-51ce-4da9-b1e2-9b474f8a877c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:32.236488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a4bd9-51ce-4da9-b1e2-9b474f8a877c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:32.244370 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a4bd9-51ce-4da9-b1e2-9b474f8a877c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:32.245824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a4bd9-51ce-4da9-b1e2-9b474f8a877c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-453a4bd9-51ce-4da9-b1e2-9b474f8a877c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:32.261048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:32.262013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:32.262990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:32.262990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:32.262990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:32.262990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:32.262990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:32.262990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:32.262990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:32.262990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:32.262990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 03:36:32.262990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:32.262990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 328/663] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:32.263886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a4bd9-51ce-4da9-b1e2-9b474f8a877c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:32.299929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595706a-00ba-4081-9344-3074f0c2554e None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:32.302998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:32.338604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2dfc4-3a35-46c9-a014-08c71751ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:32.339285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2dfc4-3a35-46c9-a014-08c71751ad02 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:32.339601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2dfc4-3a35-46c9-a014-08c71751ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:32.339907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2dfc4-3a35-46c9-a014-08c71751ad02 None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:32.340758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2dfc4-3a35-46c9-a014-08c71751ad02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:32.341070 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2dfc4-3a35-46c9-a014-08c71751ad02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:32.342565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2dfc4-3a35-46c9-a014-08c71751ad02 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:32.359479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 336/664] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:32.359479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2dfc4-3a35-46c9-a014-08c71751ad02 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:32.373348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a4bd9-51ce-4da9-b1e2-9b474f8a877c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:32.410474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f310452-f14e-44fb-bcc5-7154fa553852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:32.411110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f310452-f14e-44fb-bcc5-7154fa553852 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:32.411393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f310452-f14e-44fb-bcc5-7154fa553852 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:32.411668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f310452-f14e-44fb-bcc5-7154fa553852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:32.465582 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2dfc4-3a35-46c9-a014-08c71751ad02 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:32.477092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:32.488950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f310452-f14e-44fb-bcc5-7154fa553852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:32.490445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f310452-f14e-44fb-bcc5-7154fa553852 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:32.502323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f4a86-bea1-459a-bc78-1876e7a7e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:32.503218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f4a86-bea1-459a-bc78-1876e7a7e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:32.503506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f4a86-bea1-459a-bc78-1876e7a7e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:32.503779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f4a86-bea1-459a-bc78-1876e7a7e7f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:32.516366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f4a86-bea1-459a-bc78-1876e7a7e7f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-1f8f4a86-bea1-459a-bc78-1876e7a7e7f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:32.524053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:32.525026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:32.525957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:32.525957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:32.525957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:32.525957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:32.525957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:32.525957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:32.525957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:32.525957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:32.525957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:32.525957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:32.525957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:32.525957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:32.530153 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 337/665] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:32.530961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f4a86-bea1-459a-bc78-1876e7a7e7f8 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:32.557017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:32.595486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 329/666] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:32.596375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f310452-f14e-44fb-bcc5-7154fa553852 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:32.644813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f4a86-bea1-459a-bc78-1876e7a7e7f8 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:32.680161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b17f6b-0be1-4e2d-bc53-06fee75d818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:32.680971 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b17f6b-0be1-4e2d-bc53-06fee75d818d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:32.681257 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b17f6b-0be1-4e2d-bc53-06fee75d818d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:32.681533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b17f6b-0be1-4e2d-bc53-06fee75d818d None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:32.682366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b17f6b-0be1-4e2d-bc53-06fee75d818d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:32.682810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b17f6b-0be1-4e2d-bc53-06fee75d818d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:32.684130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b17f6b-0be1-4e2d-bc53-06fee75d818d None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:32.688618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:32.699870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f310452-f14e-44fb-bcc5-7154fa553852 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:32.706464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 338/667] 10.222.0.20 () {56 vars in 1137 bytes} [Tue Jul 12 03:36:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:32.706464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b17f6b-0be1-4e2d-bc53-06fee75d818d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:32.744251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17099ebd-dc0d-487d-8fb2-3da5676fde7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:32.745458 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17099ebd-dc0d-487d-8fb2-3da5676fde7e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:32.745747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17099ebd-dc0d-487d-8fb2-3da5676fde7e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:32.746023 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17099ebd-dc0d-487d-8fb2-3da5676fde7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:32.747368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17099ebd-dc0d-487d-8fb2-3da5676fde7e None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=7138318e53ff4b7885942d49b19eb0a5) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:32.747705 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17099ebd-dc0d-487d-8fb2-3da5676fde7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:32.749909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17099ebd-dc0d-487d-8fb2-3da5676fde7e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:32.758460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 330/668] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:32 2022] GET /identity/v3/role_assignments?user.id=7138318e53ff4b7885942d49b19eb0a5&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:32.759570 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17099ebd-dc0d-487d-8fb2-3da5676fde7e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:32.799285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:32.828872 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b17f6b-0be1-4e2d-bc53-06fee75d818d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:32.867581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7112623-4553-41f6-aea4-8f987b8e9939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:32.868190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7112623-4553-41f6-aea4-8f987b8e9939 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:32.868478 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7112623-4553-41f6-aea4-8f987b8e9939 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:32.868750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7112623-4553-41f6-aea4-8f987b8e9939 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:32.869571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7112623-4553-41f6-aea4-8f987b8e9939 None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:32.869889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7112623-4553-41f6-aea4-8f987b8e9939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:32.871324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7112623-4553-41f6-aea4-8f987b8e9939 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:32.881121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 339/669] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:32.882108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7112623-4553-41f6-aea4-8f987b8e9939 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:32.890524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17099ebd-dc0d-487d-8fb2-3da5676fde7e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:32.930831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d059a-4d35-4041-9bad-f7fa5ab7186e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:32.931633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d059a-4d35-4041-9bad-f7fa5ab7186e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:32.932152 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d059a-4d35-4041-9bad-f7fa5ab7186e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:32.932258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d059a-4d35-4041-9bad-f7fa5ab7186e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:32.948544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d059a-4d35-4041-9bad-f7fa5ab7186e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:32.950028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d059a-4d35-4041-9bad-f7fa5ab7186e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-914d059a-4d35-4041-9bad-f7fa5ab7186e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:32.974891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:32.975838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:32.976764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:32.976764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:32.976764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:32.976764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:32.976764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 331/670] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:32.977529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d059a-4d35-4041-9bad-f7fa5ab7186e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:32.988726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:33.002078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7112623-4553-41f6-aea4-8f987b8e9939 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:33.037222 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20f97c-900c-4809-9164-233259cfe762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:33.037958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20f97c-900c-4809-9164-233259cfe762 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:33.038247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20f97c-900c-4809-9164-233259cfe762 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:33.038529 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20f97c-900c-4809-9164-233259cfe762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:33.054609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:33.106617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d059a-4d35-4041-9bad-f7fa5ab7186e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:33.120629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20f97c-900c-4809-9164-233259cfe762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:33.122083 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20f97c-900c-4809-9164-233259cfe762 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:33.157073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5704223-9ecb-4f2c-8897-bd124b3b5269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:33.157927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5704223-9ecb-4f2c-8897-bd124b3b5269 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:33.158145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5704223-9ecb-4f2c-8897-bd124b3b5269 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:33.158414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5704223-9ecb-4f2c-8897-bd124b3b5269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:33.169190 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5704223-9ecb-4f2c-8897-bd124b3b5269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:33.170668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5704223-9ecb-4f2c-8897-bd124b3b5269 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-b5704223-9ecb-4f2c-8897-bd124b3b5269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:33.188851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:33.190148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:33.191441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:33.191441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:33.191441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:33.191441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:33.191441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 332/671] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:33.191441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5704223-9ecb-4f2c-8897-bd124b3b5269 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:33.242899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 340/672] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:33.243769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20f97c-900c-4809-9164-233259cfe762 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:33.275387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:33.297498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:33.322353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5704223-9ecb-4f2c-8897-bd124b3b5269 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:33.359464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb48d0-b54d-418b-b138-16dd97cc8563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:33.360310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb48d0-b54d-418b-b138-16dd97cc8563 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:33.360777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb48d0-b54d-418b-b138-16dd97cc8563 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:33.361213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb48d0-b54d-418b-b138-16dd97cc8563 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:33.362147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb48d0-b54d-418b-b138-16dd97cc8563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:33.362559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb48d0-b54d-418b-b138-16dd97cc8563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:33.364095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb48d0-b54d-418b-b138-16dd97cc8563 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:33.368991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20f97c-900c-4809-9164-233259cfe762 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:33.381071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 333/673] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:33.381071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb48d0-b54d-418b-b138-16dd97cc8563 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:33.422599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ee4ff-9d17-47f7-ae87-2f8bc8e649c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:33.423638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ee4ff-9d17-47f7-ae87-2f8bc8e649c6 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:33.424104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ee4ff-9d17-47f7-ae87-2f8bc8e649c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:33.425055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ee4ff-9d17-47f7-ae87-2f8bc8e649c6 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:33.425932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ee4ff-9d17-47f7-ae87-2f8bc8e649c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:33.426378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ee4ff-9d17-47f7-ae87-2f8bc8e649c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:33.428796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ee4ff-9d17-47f7-ae87-2f8bc8e649c6 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:33.445115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 341/674] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:33.446252 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ee4ff-9d17-47f7-ae87-2f8bc8e649c6 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:33.495478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb48d0-b54d-418b-b138-16dd97cc8563 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:33.514758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:33.540132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036934fe-c3e8-46ab-b4df-7d2a90ca111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:33.540869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036934fe-c3e8-46ab-b4df-7d2a90ca111e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:33.541240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036934fe-c3e8-46ab-b4df-7d2a90ca111e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:33.541607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036934fe-c3e8-46ab-b4df-7d2a90ca111e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:33.553343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036934fe-c3e8-46ab-b4df-7d2a90ca111e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:33.555183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036934fe-c3e8-46ab-b4df-7d2a90ca111e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-036934fe-c3e8-46ab-b4df-7d2a90ca111e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:33.574489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:33.575606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:33.576724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:33.576724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:33.576724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:33.576724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:33.577275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ee4ff-9d17-47f7-ae87-2f8bc8e649c6 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:33.578058 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 334/675] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:33.583363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036934fe-c3e8-46ab-b4df-7d2a90ca111e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:33.585791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:33.621825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb225a-91fc-4806-ac1a-89d147975042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:33.622873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb225a-91fc-4806-ac1a-89d147975042 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:33.623294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb225a-91fc-4806-ac1a-89d147975042 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:33.623767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb225a-91fc-4806-ac1a-89d147975042 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:33.632889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb225a-91fc-4806-ac1a-89d147975042 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:33.635069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb225a-91fc-4806-ac1a-89d147975042 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-07eb225a-91fc-4806-ac1a-89d147975042 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:33.654573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:33.655644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:33.656735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:33.656735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:33.656735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:33.656735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:33.656735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:33.656735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:33.656735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:33.656735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:33.663594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 342/676] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:33.663594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb225a-91fc-4806-ac1a-89d147975042 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:33.709701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036934fe-c3e8-46ab-b4df-7d2a90ca111e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:33.744087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1f356-9f10-468a-8bb1-9c2d1a5b1eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:33.745173 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1f356-9f10-468a-8bb1-9c2d1a5b1eef None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:33.745672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1f356-9f10-468a-8bb1-9c2d1a5b1eef None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:33.746132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1f356-9f10-468a-8bb1-9c2d1a5b1eef None admin] PATH_INFO: `/v3/users/swift` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:33.754456 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1f356-9f10-468a-8bb1-9c2d1a5b1eef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:33.756206 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1f356-9f10-468a-8bb1-9c2d1a5b1eef None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-14a1f356-9f10-468a-8bb1-9c2d1a5b1eef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:33.770826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:33.771766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:33.772833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:33.772833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:33.772833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:33.772833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:33.772833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:33.772833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:33.772833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:33.772833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:33.772833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 03:36:33.772833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:33.775280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 335/677] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:33.776260 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1f356-9f10-468a-8bb1-9c2d1a5b1eef None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:33.780911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb225a-91fc-4806-ac1a-89d147975042 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:33.811450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:33.825269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9766bf6-c0bb-4758-85d8-1610fb1541af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:33.826022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9766bf6-c0bb-4758-85d8-1610fb1541af None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:33.826371 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9766bf6-c0bb-4758-85d8-1610fb1541af None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:33.827093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9766bf6-c0bb-4758-85d8-1610fb1541af None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:33.828034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9766bf6-c0bb-4758-85d8-1610fb1541af None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:33.828549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9766bf6-c0bb-4758-85d8-1610fb1541af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:33.830278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9766bf6-c0bb-4758-85d8-1610fb1541af None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:33.848048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 343/678] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:33.848048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9766bf6-c0bb-4758-85d8-1610fb1541af None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:33.880039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:33.887817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1f356-9f10-468a-8bb1-9c2d1a5b1eef None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:33.922676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e83f3-d7f6-42ed-bdb5-a4f51bc5f4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:33.923360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e83f3-d7f6-42ed-bdb5-a4f51bc5f4ec None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:33.923641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e83f3-d7f6-42ed-bdb5-a4f51bc5f4ec None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:33.923910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e83f3-d7f6-42ed-bdb5-a4f51bc5f4ec None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:33.924714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e83f3-d7f6-42ed-bdb5-a4f51bc5f4ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:33.925010 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e83f3-d7f6-42ed-bdb5-a4f51bc5f4ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:33.926457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e83f3-d7f6-42ed-bdb5-a4f51bc5f4ec None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:33.940126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 336/679] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:33.941811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c36232-ab81-44dc-8ee9-c052573358cb None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:33.941811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c36232-ab81-44dc-8ee9-c052573358cb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:33.941811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c36232-ab81-44dc-8ee9-c052573358cb None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:33.947000 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 337/680] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:33.947695 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c36232-ab81-44dc-8ee9-c052573358cb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:33.971484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9766bf6-c0bb-4758-85d8-1610fb1541af None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:34.007494 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cc3cf-d8d1-4e64-80af-36ce34621a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:34.010206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cc3cf-d8d1-4e64-80af-36ce34621a0d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:34.010459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cc3cf-d8d1-4e64-80af-36ce34621a0d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:34.010708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cc3cf-d8d1-4e64-80af-36ce34621a0d None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:34.011588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cc3cf-d8d1-4e64-80af-36ce34621a0d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:34.011898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cc3cf-d8d1-4e64-80af-36ce34621a0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:34.013764 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cc3cf-d8d1-4e64-80af-36ce34621a0d None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:34.033292 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 344/681] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Jul 12 03:36:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:34.038142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cc3cf-d8d1-4e64-80af-36ce34621a0d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:34.050226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:34.061924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c36232-ab81-44dc-8ee9-c052573358cb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:34.096274 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b79236-7751-4fe6-b860-a0b8f9b35c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:34.097069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b79236-7751-4fe6-b860-a0b8f9b35c5c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:34.097348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b79236-7751-4fe6-b860-a0b8f9b35c5c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:34.097621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b79236-7751-4fe6-b860-a0b8f9b35c5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:34.104508 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b79236-7751-4fe6-b860-a0b8f9b35c5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:34.105963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b79236-7751-4fe6-b860-a0b8f9b35c5c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a0b79236-7751-4fe6-b860-a0b8f9b35c5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:34.119731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:34.120765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:34.121639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:34.121639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:34.121639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:34.121639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:34.121639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:34.121639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:34.121639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:34.121639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:34.121639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 338/682] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:34.122712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3469227-35f7-4261-a383-dde53df0faf1 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:34.122712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3469227-35f7-4261-a383-dde53df0faf1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:34.122712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3469227-35f7-4261-a383-dde53df0faf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:34.127311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:34.139474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a3469227-35f7-4261-a383-dde53df0faf1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:34.140689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a3469227-35f7-4261-a383-dde53df0faf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:34.147742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cc3cf-d8d1-4e64-80af-36ce34621a0d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:34.181806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a667b50-1b9a-4c98-bac9-e7c5e4184b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:34.182455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a667b50-1b9a-4c98-bac9-e7c5e4184b98 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:34.182736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a667b50-1b9a-4c98-bac9-e7c5e4184b98 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:34.183015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a667b50-1b9a-4c98-bac9-e7c5e4184b98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:34.183833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a667b50-1b9a-4c98-bac9-e7c5e4184b98 None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=139be21e7f644b5796d30158851ebd00) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:34.184191 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a667b50-1b9a-4c98-bac9-e7c5e4184b98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:34.185621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a667b50-1b9a-4c98-bac9-e7c5e4184b98 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:34.191820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 345/683] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:34 2022] GET /identity/v3/role_assignments?user.id=139be21e7f644b5796d30158851ebd00&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:34.197241 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a667b50-1b9a-4c98-bac9-e7c5e4184b98 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:34.216481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 339/684] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:34.217808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3469227-35f7-4261-a383-dde53df0faf1 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:34.330149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:34.331832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a667b50-1b9a-4c98-bac9-e7c5e4184b98 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:34.360269 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3469227-35f7-4261-a383-dde53df0faf1 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:34.372945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30ed4f-13f5-4372-97c7-fd3f42ca3828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:34.373633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30ed4f-13f5-4372-97c7-fd3f42ca3828 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:34.373911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30ed4f-13f5-4372-97c7-fd3f42ca3828 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:34.374186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30ed4f-13f5-4372-97c7-fd3f42ca3828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:34.391762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:34.404512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa48d4-c4d4-4b2b-a2de-3636624337d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:34.405198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa48d4-c4d4-4b2b-a2de-3636624337d2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:34.405479 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa48d4-c4d4-4b2b-a2de-3636624337d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:34.405747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa48d4-c4d4-4b2b-a2de-3636624337d2 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:34.406547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa48d4-c4d4-4b2b-a2de-3636624337d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:34.406850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa48d4-c4d4-4b2b-a2de-3636624337d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:34.409189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa48d4-c4d4-4b2b-a2de-3636624337d2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:34.423555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 340/685] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:34.424486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa48d4-c4d4-4b2b-a2de-3636624337d2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:34.459380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30ed4f-13f5-4372-97c7-fd3f42ca3828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:34.462797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30ed4f-13f5-4372-97c7-fd3f42ca3828 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:34.538803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa48d4-c4d4-4b2b-a2de-3636624337d2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:34.573863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6c77-5070-42ee-a595-8f6587d0bffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:34.574073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 346/686] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:34.574720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6c77-5070-42ee-a595-8f6587d0bffe None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:34.574901 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30ed4f-13f5-4372-97c7-fd3f42ca3828 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:34.574996 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6c77-5070-42ee-a595-8f6587d0bffe None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:34.575266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6c77-5070-42ee-a595-8f6587d0bffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:34.584813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:34.586988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc6c77-5070-42ee-a595-8f6587d0bffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:34.588532 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc6c77-5070-42ee-a595-8f6587d0bffe None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-eedc6c77-5070-42ee-a595-8f6587d0bffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:34.606717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:34.607658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:34.608561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:34.608561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:34.608561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:34.608561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:34.612576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 341/687] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:34.613455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6c77-5070-42ee-a595-8f6587d0bffe None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:34.625498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:34.684095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30ed4f-13f5-4372-97c7-fd3f42ca3828 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:34.717222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6c77-5070-42ee-a595-8f6587d0bffe None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:34.721917 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7282b3-c63b-4692-9fa2-e82145a93e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:34.722621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7282b3-c63b-4692-9fa2-e82145a93e05 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:34.722905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7282b3-c63b-4692-9fa2-e82145a93e05 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:34.723178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7282b3-c63b-4692-9fa2-e82145a93e05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:34.724034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7282b3-c63b-4692-9fa2-e82145a93e05 None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=cee6da907b614dc5b6ffb47e2860f459) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:34.724338 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7282b3-c63b-4692-9fa2-e82145a93e05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:34.725919 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7282b3-c63b-4692-9fa2-e82145a93e05 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:34.732949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 347/688] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:34 2022] GET /identity/v3/role_assignments?user.id=cee6da907b614dc5b6ffb47e2860f459&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:34.733815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7282b3-c63b-4692-9fa2-e82145a93e05 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:34.786362 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f611df-fa70-4af9-85b7-ee3058b08220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:34.787687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f611df-fa70-4af9-85b7-ee3058b08220 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:34.788191 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f611df-fa70-4af9-85b7-ee3058b08220 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:34.788660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f611df-fa70-4af9-85b7-ee3058b08220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:34.806034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f611df-fa70-4af9-85b7-ee3058b08220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:34.807877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f611df-fa70-4af9-85b7-ee3058b08220 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e9f611df-fa70-4af9-85b7-ee3058b08220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:34.841794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:34.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:34.843634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:34.843634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:34.843634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:34.843634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:34.847648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 342/689] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:34.855595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f611df-fa70-4af9-85b7-ee3058b08220 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:34.920577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:34.927011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7282b3-c63b-4692-9fa2-e82145a93e05 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:34.945437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:34.980843 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33642d2c-513c-41d4-a666-e259aa610790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:34.981516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33642d2c-513c-41d4-a666-e259aa610790 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:34.981796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33642d2c-513c-41d4-a666-e259aa610790 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:34.982069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33642d2c-513c-41d4-a666-e259aa610790 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:34.983212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33642d2c-513c-41d4-a666-e259aa610790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:34.983212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33642d2c-513c-41d4-a666-e259aa610790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:34.984625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33642d2c-513c-41d4-a666-e259aa610790 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:34.998988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f611df-fa70-4af9-85b7-ee3058b08220 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:35.004023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 348/690] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:35.004023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a507817-bd6b-41f1-a58d-e718572ba1eb None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:35.004023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a507817-bd6b-41f1-a58d-e718572ba1eb None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:35.004023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a507817-bd6b-41f1-a58d-e718572ba1eb None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:35.004023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 349/691] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:35.004741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a507817-bd6b-41f1-a58d-e718572ba1eb None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:35.047332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c486cdc-7545-4b83-8c22-3de347326163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:35.050760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c486cdc-7545-4b83-8c22-3de347326163 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:35.050760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c486cdc-7545-4b83-8c22-3de347326163 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:35.050760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c486cdc-7545-4b83-8c22-3de347326163 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:35.050760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c486cdc-7545-4b83-8c22-3de347326163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:35.050760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c486cdc-7545-4b83-8c22-3de347326163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:35.052800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c486cdc-7545-4b83-8c22-3de347326163 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:35.072088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 343/692] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:35.074539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ab891-3b7b-453d-bfd3-37f60bbed2b6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:35.074823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ab891-3b7b-453d-bfd3-37f60bbed2b6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:35.075091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ab891-3b7b-453d-bfd3-37f60bbed2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:35.092020 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-122ab891-3b7b-453d-bfd3-37f60bbed2b6 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:35.093143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-122ab891-3b7b-453d-bfd3-37f60bbed2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:35.124647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a507817-bd6b-41f1-a58d-e718572ba1eb None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:35.169029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fcb5a-f64f-430b-b129-d230e21a2d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:35.169698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fcb5a-f64f-430b-b129-d230e21a2d53 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:35.169983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fcb5a-f64f-430b-b129-d230e21a2d53 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:35.170253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fcb5a-f64f-430b-b129-d230e21a2d53 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:35.170961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fcb5a-f64f-430b-b129-d230e21a2d53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:35.172444 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2fcb5a-f64f-430b-b129-d230e21a2d53 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:35.173441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 344/693] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:35.174676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ab891-3b7b-453d-bfd3-37f60bbed2b6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:35.207256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 350/694] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:36:35 2022] POST /identity/v3/users => generated 254 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:36:35.211511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fcb5a-f64f-430b-b129-d230e21a2d53 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:35.221405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:35.262038 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:35.305910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ab891-3b7b-453d-bfd3-37f60bbed2b6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:35.348609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95b118-6ce6-45a0-8b10-9a5066199ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:35.350799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95b118-6ce6-45a0-8b10-9a5066199ecb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:35.351303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95b118-6ce6-45a0-8b10-9a5066199ecb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:35.351787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95b118-6ce6-45a0-8b10-9a5066199ecb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:35.360814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95b118-6ce6-45a0-8b10-9a5066199ecb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:35.363313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95b118-6ce6-45a0-8b10-9a5066199ecb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-0e95b118-6ce6-45a0-8b10-9a5066199ecb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:35.381681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:35.382635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:35.383523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:35.383523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:35.383523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:35.383523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:35.383523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:35.383523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:35.383523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:35.383523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:35.391442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 345/695] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:35.397094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95b118-6ce6-45a0-8b10-9a5066199ecb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:35.435030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fcb5a-f64f-430b-b129-d230e21a2d53 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:35.494277 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72c875-ec1d-48dc-8e1b-ccae4e9d16e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:35.495134 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72c875-ec1d-48dc-8e1b-ccae4e9d16e5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:35.495418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72c875-ec1d-48dc-8e1b-ccae4e9d16e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:35.495688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72c875-ec1d-48dc-8e1b-ccae4e9d16e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:35.512086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b72c875-ec1d-48dc-8e1b-ccae4e9d16e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-6b72c875-ec1d-48dc-8e1b-ccae4e9d16e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:35.519951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:35.520894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:35.521724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:35.521724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:35.521724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:35.521724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:35.521724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:35.521724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:35.521724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:35.521724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:35.521724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:35.521724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:35.521724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:35.521724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:35.522593 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 351/696] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:35.525738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:35.528026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72c875-ec1d-48dc-8e1b-ccae4e9d16e5 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:35.534725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95b118-6ce6-45a0-8b10-9a5066199ecb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:35.583527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bca5d8-b404-41e6-894a-c5e293f2a3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:35.584790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bca5d8-b404-41e6-894a-c5e293f2a3a3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:35.585222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bca5d8-b404-41e6-894a-c5e293f2a3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:35.585865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bca5d8-b404-41e6-894a-c5e293f2a3a3 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:35.587239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bca5d8-b404-41e6-894a-c5e293f2a3a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:35.587720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bca5d8-b404-41e6-894a-c5e293f2a3a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:35.589394 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bca5d8-b404-41e6-894a-c5e293f2a3a3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:35.607297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 346/697] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:35.613380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bca5d8-b404-41e6-894a-c5e293f2a3a3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:35.638898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:35.660600 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72c875-ec1d-48dc-8e1b-ccae4e9d16e5 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:35.706809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7766d-3a37-48b1-8594-2dc4b27a7ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:35.707739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7766d-3a37-48b1-8594-2dc4b27a7ad3 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:35.708195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7766d-3a37-48b1-8594-2dc4b27a7ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:35.709043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7766d-3a37-48b1-8594-2dc4b27a7ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:35.709979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7766d-3a37-48b1-8594-2dc4b27a7ad3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:35.710807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7766d-3a37-48b1-8594-2dc4b27a7ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:35.712654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7766d-3a37-48b1-8594-2dc4b27a7ad3 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:35.724762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 352/698] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:35.732944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7766d-3a37-48b1-8594-2dc4b27a7ad3 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:35.735090 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bca5d8-b404-41e6-894a-c5e293f2a3a3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:35.772219 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcb473-3689-4983-b8fb-c40a68b581e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:35.773019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcb473-3689-4983-b8fb-c40a68b581e6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:35.773336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcb473-3689-4983-b8fb-c40a68b581e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:35.773633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcb473-3689-4983-b8fb-c40a68b581e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:35.774564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcb473-3689-4983-b8fb-c40a68b581e6 None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=0026b8f11158492b90da3395cc2c43c6) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:35.774884 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcb473-3689-4983-b8fb-c40a68b581e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:35.776464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcb473-3689-4983-b8fb-c40a68b581e6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:35.788147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 347/699] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:35 2022] GET /identity/v3/role_assignments?user.id=0026b8f11158492b90da3395cc2c43c6&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:35.847657 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7766d-3a37-48b1-8594-2dc4b27a7ad3 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:35.891531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acb475-ae7c-4954-a387-1aeec3747475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:35.892943 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acb475-ae7c-4954-a387-1aeec3747475 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:35.893427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acb475-ae7c-4954-a387-1aeec3747475 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:35.893909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acb475-ae7c-4954-a387-1aeec3747475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:35.906021 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acb475-ae7c-4954-a387-1aeec3747475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:35.907811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acb475-ae7c-4954-a387-1aeec3747475 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:35.921254 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-a2acb475-ae7c-4954-a387-1aeec3747475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:35.936391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:35.937348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:35.938220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:35.938220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:35.938220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:35.938220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:35.940678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 353/700] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:35.948418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcb473-3689-4983-b8fb-c40a68b581e6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:35.987728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:36.082783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcb473-3689-4983-b8fb-c40a68b581e6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:36.125802 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11636d47-d059-4274-b712-ccbd98c6aafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:36.126571 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11636d47-d059-4274-b712-ccbd98c6aafe None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:36.126855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11636d47-d059-4274-b712-ccbd98c6aafe None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:36.127132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11636d47-d059-4274-b712-ccbd98c6aafe None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:36.127899 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11636d47-d059-4274-b712-ccbd98c6aafe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:36.128242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11636d47-d059-4274-b712-ccbd98c6aafe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:36.129639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11636d47-d059-4274-b712-ccbd98c6aafe None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:36.144732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 348/701] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:36.149999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acb475-ae7c-4954-a387-1aeec3747475 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:36.288421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acb475-ae7c-4954-a387-1aeec3747475 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:36.322754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f49b6-512c-4bb9-af90-e23913ba134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:36.323553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f49b6-512c-4bb9-af90-e23913ba134f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:36.323834 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f49b6-512c-4bb9-af90-e23913ba134f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:36.324142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f49b6-512c-4bb9-af90-e23913ba134f None admin] PATH_INFO: `/v3/users/nova` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:36.331848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f49b6-512c-4bb9-af90-e23913ba134f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:36.333332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f49b6-512c-4bb9-af90-e23913ba134f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-633f49b6-512c-4bb9-af90-e23913ba134f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:36.353697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:36.354693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:36.355611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:36.355611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:36.355611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:36.355611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:36.355611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:36.355611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:36.355611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:36.355611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:36.355611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:36:36.355611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:36.355611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 354/702] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 12 03:36:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:36.365138 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11636d47-d059-4274-b712-ccbd98c6aafe None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:36.456702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:36.479568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11636d47-d059-4274-b712-ccbd98c6aafe None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:36.494046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2e430-978a-4133-8a0e-7da20b57219c None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:36.494396 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2e430-978a-4133-8a0e-7da20b57219c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:36.494609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2e430-978a-4133-8a0e-7da20b57219c None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:36.500114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 355/703] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:36.501334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa74511-1a6c-45e4-98e3-48c5a4155134 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:36.501622 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa74511-1a6c-45e4-98e3-48c5a4155134 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:36.501896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa74511-1a6c-45e4-98e3-48c5a4155134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:36.520726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2898d-f2b7-41de-a05f-9819e85cb037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:36.521745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2898d-f2b7-41de-a05f-9819e85cb037 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:36.522244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2898d-f2b7-41de-a05f-9819e85cb037 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:36.522709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2898d-f2b7-41de-a05f-9819e85cb037 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:36.524215 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2898d-f2b7-41de-a05f-9819e85cb037 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:36.524755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2898d-f2b7-41de-a05f-9819e85cb037 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:36.526474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2898d-f2b7-41de-a05f-9819e85cb037 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:36.537110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-5fa74511-1a6c-45e4-98e3-48c5a4155134 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:36.544029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5fa74511-1a6c-45e4-98e3-48c5a4155134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:36.562595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 349/704] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 12 03:36:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:36.568765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2898d-f2b7-41de-a05f-9819e85cb037 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:36.647865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:36.668964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 356/705] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:36.701221 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa74511-1a6c-45e4-98e3-48c5a4155134 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:36.735475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2898d-f2b7-41de-a05f-9819e85cb037 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:36.804973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6bb69-d129-4935-b66a-073a90f9c21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:36.805623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6bb69-d129-4935-b66a-073a90f9c21a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:36.805901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6bb69-d129-4935-b66a-073a90f9c21a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:36.806168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6bb69-d129-4935-b66a-073a90f9c21a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:36.815800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:36.878436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa74511-1a6c-45e4-98e3-48c5a4155134 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:36.891537 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6bb69-d129-4935-b66a-073a90f9c21a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:36.893062 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6bb69-d129-4935-b66a-073a90f9c21a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:36.992891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22221bd9-a976-45ed-bc82-e05301c38bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:36.993659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22221bd9-a976-45ed-bc82-e05301c38bd7 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:36.993956 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22221bd9-a976-45ed-bc82-e05301c38bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:36.994234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22221bd9-a976-45ed-bc82-e05301c38bd7 None admin] PATH_INFO: `/v3/services/network` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:36.994939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22221bd9-a976-45ed-bc82-e05301c38bd7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:36.996421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22221bd9-a976-45ed-bc82-e05301c38bd7 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:37.008101 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 350/706] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:37.009535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6bb69-d129-4935-b66a-073a90f9c21a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-22221bd9-a976-45ed-bc82-e05301c38bd7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:37.034186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:37.035375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:37.036298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 357/707] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:36 2022] GET /identity/v3/services/network => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:37.044439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:37.044966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22221bd9-a976-45ed-bc82-e05301c38bd7 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:37.110653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6bb69-d129-4935-b66a-073a90f9c21a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:37.145486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fa268-6b96-4c62-87f9-0939ec7b5c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:37.146311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fa268-6b96-4c62-87f9-0939ec7b5c1c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:37.146634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fa268-6b96-4c62-87f9-0939ec7b5c1c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:37.146906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fa268-6b96-4c62-87f9-0939ec7b5c1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:37.153743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:37.157462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fa268-6b96-4c62-87f9-0939ec7b5c1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:37.159036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fa268-6b96-4c62-87f9-0939ec7b5c1c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e26fa268-6b96-4c62-87f9-0939ec7b5c1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:37.176976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:37.179088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:37.181239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:37.181239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:37.181239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:37.181239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:37.181239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 351/708] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:37.184908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fa268-6b96-4c62-87f9-0939ec7b5c1c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:37.256770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22221bd9-a976-45ed-bc82-e05301c38bd7 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:37.288313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:37.301059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fa268-6b96-4c62-87f9-0939ec7b5c1c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:37.303291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1aeb44-7484-4a35-aff6-5741d1b8afa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:37.304098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1aeb44-7484-4a35-aff6-5741d1b8afa4 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:37.304389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1aeb44-7484-4a35-aff6-5741d1b8afa4 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:37.304669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1aeb44-7484-4a35-aff6-5741d1b8afa4 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:37.305384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1aeb44-7484-4a35-aff6-5741d1b8afa4 None admin] RBAC: Adding query filter params (name=network) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:37.305694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1aeb44-7484-4a35-aff6-5741d1b8afa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:37.307089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1aeb44-7484-4a35-aff6-5741d1b8afa4 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:37.312273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 358/709] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:36:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:37.313713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55361f4-abd1-4d8f-99bf-7aeb1c83285c None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:37.314002 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55361f4-abd1-4d8f-99bf-7aeb1c83285c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:37.314283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55361f4-abd1-4d8f-99bf-7aeb1c83285c None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:37.316055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 359/710] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:37.317504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55361f4-abd1-4d8f-99bf-7aeb1c83285c None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:37.336205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790f924-94a3-4926-9b78-9cd7c2fc2fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:37.336869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790f924-94a3-4926-9b78-9cd7c2fc2fbf None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:37.337146 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790f924-94a3-4926-9b78-9cd7c2fc2fbf None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:37.337418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790f924-94a3-4926-9b78-9cd7c2fc2fbf None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:37.338134 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790f924-94a3-4926-9b78-9cd7c2fc2fbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:37.338437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790f924-94a3-4926-9b78-9cd7c2fc2fbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:37.339810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790f924-94a3-4926-9b78-9cd7c2fc2fbf None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:37.351916 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 352/711] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:37.357983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ffe41-2470-4b02-b5c4-f91d52c4d304 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:37.358276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ffe41-2470-4b02-b5c4-f91d52c4d304 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:37.358550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ffe41-2470-4b02-b5c4-f91d52c4d304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:37.375011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-971ffe41-2470-4b02-b5c4-f91d52c4d304 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:37.376449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-971ffe41-2470-4b02-b5c4-f91d52c4d304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:37.437150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55361f4-abd1-4d8f-99bf-7aeb1c83285c None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:37.450750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 353/712] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:37.451635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ffe41-2470-4b02-b5c4-f91d52c4d304 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:37.459486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:37.475051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d137bb-cee1-4590-be0c-44309af7c3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:37.475744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d137bb-cee1-4590-be0c-44309af7c3a1 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:37.476059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d137bb-cee1-4590-be0c-44309af7c3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:37.476361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d137bb-cee1-4590-be0c-44309af7c3a1 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:37.485707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d137bb-cee1-4590-be0c-44309af7c3a1 None admin] RBAC: Adding query filter params (type=network) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:37.486167 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d137bb-cee1-4590-be0c-44309af7c3a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:37.487701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d137bb-cee1-4590-be0c-44309af7c3a1 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:37.500600 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 360/713] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:36:37 2022] GET /identity/v3/services?type=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:37.501789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d137bb-cee1-4590-be0c-44309af7c3a1 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:37.537884 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:37.559278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ffe41-2470-4b02-b5c4-f91d52c4d304 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:37.618704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fa45f-7c90-42f6-90b8-4e3ac0ff97a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:37.619518 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fa45f-7c90-42f6-90b8-4e3ac0ff97a3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:37.619797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fa45f-7c90-42f6-90b8-4e3ac0ff97a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:37.620104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fa45f-7c90-42f6-90b8-4e3ac0ff97a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:37.628112 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05fa45f-7c90-42f6-90b8-4e3ac0ff97a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:37.629624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05fa45f-7c90-42f6-90b8-4e3ac0ff97a3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c05fa45f-7c90-42f6-90b8-4e3ac0ff97a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:37.648427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:37.649520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:37.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:37.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:37.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:37.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:37.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:37.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:37.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:37.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:37.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 354/714] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:37.656102 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fa45f-7c90-42f6-90b8-4e3ac0ff97a3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:37.712508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d137bb-cee1-4590-be0c-44309af7c3a1 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:37.770774 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cec93-3183-4ff4-9e9b-4f091f70cf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:37.772134 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cec93-3183-4ff4-9e9b-4f091f70cf1e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:37.772425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cec93-3183-4ff4-9e9b-4f091f70cf1e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:37.772707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206cec93-3183-4ff4-9e9b-4f091f70cf1e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:37.773374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206cec93-3183-4ff4-9e9b-4f091f70cf1e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:37.775305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206cec93-3183-4ff4-9e9b-4f091f70cf1e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-206cec93-3183-4ff4-9e9b-4f091f70cf1e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:37.792467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:37.793456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:37.794304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 361/715] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Jul 12 03:36:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:37.795288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fa45f-7c90-42f6-90b8-4e3ac0ff97a3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:37.802550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cec93-3183-4ff4-9e9b-4f091f70cf1e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:37.854049 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ddb69-7001-440a-9489-3abb89ef0b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:37.855178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ddb69-7001-440a-9489-3abb89ef0b7b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:37.856585 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ddb69-7001-440a-9489-3abb89ef0b7b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:37.857137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ddb69-7001-440a-9489-3abb89ef0b7b None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:37.858212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ddb69-7001-440a-9489-3abb89ef0b7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:37.859442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ddb69-7001-440a-9489-3abb89ef0b7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:37.863039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ddb69-7001-440a-9489-3abb89ef0b7b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:37.873046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:37.879303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 355/716] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:37.881007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b6e7-ea7f-4a05-aae1-3e0a8b8255a2 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:37.881634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b6e7-ea7f-4a05-aae1-3e0a8b8255a2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:37.881921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b6e7-ea7f-4a05-aae1-3e0a8b8255a2 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:37.883388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 356/717] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:37.884810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397b6e7-ea7f-4a05-aae1-3e0a8b8255a2 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:37.888103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:37.963400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206cec93-3183-4ff4-9e9b-4f091f70cf1e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:37.998832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42c072-47b4-48d9-8a1c-d19dc1303f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:37.999501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42c072-47b4-48d9-8a1c-d19dc1303f06 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:37.999809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42c072-47b4-48d9-8a1c-d19dc1303f06 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:38.000129 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42c072-47b4-48d9-8a1c-d19dc1303f06 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:38.000862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42c072-47b4-48d9-8a1c-d19dc1303f06 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:38.001221 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42c072-47b4-48d9-8a1c-d19dc1303f06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:38.002659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42c072-47b4-48d9-8a1c-d19dc1303f06 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:38.006408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397b6e7-ea7f-4a05-aae1-3e0a8b8255a2 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:38.011876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 362/718] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Jul 12 03:36:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:38.011876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3ec23-4286-4974-8d7f-52893a98b3b7 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:38.011876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3ec23-4286-4974-8d7f-52893a98b3b7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:38.011876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3ec23-4286-4974-8d7f-52893a98b3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:38.033561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-5bf3ec23-4286-4974-8d7f-52893a98b3b7 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:38.035116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5bf3ec23-4286-4974-8d7f-52893a98b3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:38.052906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb89170-869d-4db6-afb3-ba6f59bb757b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:38.053573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb89170-869d-4db6-afb3-ba6f59bb757b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:38.053903 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb89170-869d-4db6-afb3-ba6f59bb757b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:38.054184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb89170-869d-4db6-afb3-ba6f59bb757b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:38.055118 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb89170-869d-4db6-afb3-ba6f59bb757b None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=7138318e53ff4b7885942d49b19eb0a5) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:38.055308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb89170-869d-4db6-afb3-ba6f59bb757b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:38.056896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb89170-869d-4db6-afb3-ba6f59bb757b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:38.072378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 357/719] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:37 2022] GET /identity/v3/role_assignments?user.id=7138318e53ff4b7885942d49b19eb0a5&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:38.073407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb89170-869d-4db6-afb3-ba6f59bb757b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:38.133301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 363/720] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:38.176445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:38.186049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3ec23-4286-4974-8d7f-52893a98b3b7 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:38.256722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb89170-869d-4db6-afb3-ba6f59bb757b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:38.259495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:38.293752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643927b-d1f2-49ef-ae0f-6f36389e7a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:38.294481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643927b-d1f2-49ef-ae0f-6f36389e7a30 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:38.294767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643927b-d1f2-49ef-ae0f-6f36389e7a30 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:38.295043 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643927b-d1f2-49ef-ae0f-6f36389e7a30 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:38.295797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643927b-d1f2-49ef-ae0f-6f36389e7a30 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:38.296146 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643927b-d1f2-49ef-ae0f-6f36389e7a30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:38.297755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643927b-d1f2-49ef-ae0f-6f36389e7a30 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:38.304759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 358/721] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Jul 12 03:36:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:38.414347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3ec23-4286-4974-8d7f-52893a98b3b7 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:38.464877 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22064c-f3b0-4758-8fa0-0038123c08f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:38.465706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22064c-f3b0-4758-8fa0-0038123c08f8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:38.465987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22064c-f3b0-4758-8fa0-0038123c08f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:38.466262 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22064c-f3b0-4758-8fa0-0038123c08f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:38.479127 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc22064c-f3b0-4758-8fa0-0038123c08f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-bc22064c-f3b0-4758-8fa0-0038123c08f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:38.486806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:38.487767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:38.488652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 364/722] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:38.494037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643927b-d1f2-49ef-ae0f-6f36389e7a30 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:38.620633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643927b-d1f2-49ef-ae0f-6f36389e7a30 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:38.649706 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:38.659105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2165eb6-01bc-426b-8fe9-09d2bd4212db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:38.660310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2165eb6-01bc-426b-8fe9-09d2bd4212db None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:38.661169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2165eb6-01bc-426b-8fe9-09d2bd4212db None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:38.661450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2165eb6-01bc-426b-8fe9-09d2bd4212db None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:38.662213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2165eb6-01bc-426b-8fe9-09d2bd4212db None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:38.662522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2165eb6-01bc-426b-8fe9-09d2bd4212db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:38.663929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2165eb6-01bc-426b-8fe9-09d2bd4212db None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:38.673040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 359/723] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:38.678559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22064c-f3b0-4758-8fa0-0038123c08f8 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:38.704051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ae4cd-42bb-4f5b-8a6c-7349a30a6f00 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:38.704389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ae4cd-42bb-4f5b-8a6c-7349a30a6f00 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:38.704624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ae4cd-42bb-4f5b-8a6c-7349a30a6f00 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:38.706252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 360/724] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:38.718415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602a339-6100-42d2-a8e5-fbb50524af5f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:38.718916 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602a339-6100-42d2-a8e5-fbb50524af5f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:38.719382 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602a339-6100-42d2-a8e5-fbb50524af5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:38.748563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1602a339-6100-42d2-a8e5-fbb50524af5f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:38.750098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1602a339-6100-42d2-a8e5-fbb50524af5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:38.847899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:38.852112 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22064c-f3b0-4758-8fa0-0038123c08f8 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:38.854836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 361/725] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:38.893990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602a339-6100-42d2-a8e5-fbb50524af5f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:38.903012 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcea8c-7fce-4f22-8114-8128985e85fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:38.903843 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcea8c-7fce-4f22-8114-8128985e85fe None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:38.904160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcea8c-7fce-4f22-8114-8128985e85fe None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:38.904435 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dcea8c-7fce-4f22-8114-8128985e85fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:38.929400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcea8c-7fce-4f22-8114-8128985e85fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:38.930882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dcea8c-7fce-4f22-8114-8128985e85fe None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-63dcea8c-7fce-4f22-8114-8128985e85fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:38.960784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:38.962017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:38.963267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:38.963267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:38.963267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:38.963267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:38.963267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 365/726] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:38.968200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcea8c-7fce-4f22-8114-8128985e85fe None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:39.020971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602a339-6100-42d2-a8e5-fbb50524af5f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:39.041547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:39.060468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b1876-e819-41d0-8347-c1beb0c28971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:39.061344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b1876-e819-41d0-8347-c1beb0c28971 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:39.061626 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b1876-e819-41d0-8347-c1beb0c28971 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:39.061899 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b1876-e819-41d0-8347-c1beb0c28971 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:39.080824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b1876-e819-41d0-8347-c1beb0c28971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-018b1876-e819-41d0-8347-c1beb0c28971 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:39.091661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:39.092630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:39.093490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:39.093490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:39.093490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:39.093490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:39.093490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:39.093490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:39.093490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:39.093490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:39.093490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:39.093490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:39.093490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:39.093490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:39.094767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 362/727] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:39.102278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b1876-e819-41d0-8347-c1beb0c28971 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:39.199889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dcea8c-7fce-4f22-8114-8128985e85fe None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:39.265131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286e52d-d104-4a8b-91cd-b62a4247e21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:39.268550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286e52d-d104-4a8b-91cd-b62a4247e21a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:39.268849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286e52d-d104-4a8b-91cd-b62a4247e21a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:39.269116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286e52d-d104-4a8b-91cd-b62a4247e21a None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:39.269846 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286e52d-d104-4a8b-91cd-b62a4247e21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:39.270152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286e52d-d104-4a8b-91cd-b62a4247e21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:39.271533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286e52d-d104-4a8b-91cd-b62a4247e21a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:39.291872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b1876-e819-41d0-8347-c1beb0c28971 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:39.298609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 366/728] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:39.302591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286e52d-d104-4a8b-91cd-b62a4247e21a None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:39.328520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:39.367736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fdb528-029c-4cd9-b559-19c46a164652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:39.368487 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdb528-029c-4cd9-b559-19c46a164652 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:39.368776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdb528-029c-4cd9-b559-19c46a164652 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:39.371962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdb528-029c-4cd9-b559-19c46a164652 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:39.372759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fdb528-029c-4cd9-b559-19c46a164652 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:39.373069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fdb528-029c-4cd9-b559-19c46a164652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:39.374477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fdb528-029c-4cd9-b559-19c46a164652 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:39.393913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 363/729] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:39.417187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fdb528-029c-4cd9-b559-19c46a164652 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:39.472438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:39.522201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286e52d-d104-4a8b-91cd-b62a4247e21a None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:39.539533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fdb528-029c-4cd9-b559-19c46a164652 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:39.578603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4bf676-3a9e-40c3-b0d6-ce0721577216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:39.579445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bf676-3a9e-40c3-b0d6-ce0721577216 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:39.579720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bf676-3a9e-40c3-b0d6-ce0721577216 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:39.579989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bf676-3a9e-40c3-b0d6-ce0721577216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:39.596962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4bf676-3a9e-40c3-b0d6-ce0721577216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:39.598514 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4bf676-3a9e-40c3-b0d6-ce0721577216 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:39.603010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca120df3-8887-4ad6-a481-9e6d1af8dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:39.603889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca120df3-8887-4ad6-a481-9e6d1af8dc0b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:39.605469 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca120df3-8887-4ad6-a481-9e6d1af8dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:39.605755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca120df3-8887-4ad6-a481-9e6d1af8dc0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:39.617066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca120df3-8887-4ad6-a481-9e6d1af8dc0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:39.618864 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca120df3-8887-4ad6-a481-9e6d1af8dc0b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-ca120df3-8887-4ad6-a481-9e6d1af8dc0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:39.646440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:39.647457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:39.648460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:39.648460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:39.648460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:39.648460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:39.648460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:39.648460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:39.648460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:39.648460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:39.648460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 03:36:39.648460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-ab4bf676-3a9e-40c3-b0d6-ce0721577216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:39.649905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:39.650738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:39.651658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:39.651658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:39.651658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:39.651658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:39.653761 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 364/730] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:39.655677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 367/731] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 12 03:36:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:39.660825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4bf676-3a9e-40c3-b0d6-ce0721577216 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:39.662087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca120df3-8887-4ad6-a481-9e6d1af8dc0b None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:39.762116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:39.810963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca120df3-8887-4ad6-a481-9e6d1af8dc0b None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:39.820252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:39.842012 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4bf676-3a9e-40c3-b0d6-ce0721577216 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:39.864920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bdd57-3d38-44af-a841-0fa6be74c316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:39.865874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bdd57-3d38-44af-a841-0fa6be74c316 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:39.867472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bdd57-3d38-44af-a841-0fa6be74c316 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:39.867899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bdd57-3d38-44af-a841-0fa6be74c316 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:39.868926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bdd57-3d38-44af-a841-0fa6be74c316 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:39.870166 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bdd57-3d38-44af-a841-0fa6be74c316 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:39.873635 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bdd57-3d38-44af-a841-0fa6be74c316 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:39.904049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 368/732] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Jul 12 03:36:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:39.910049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bdd57-3d38-44af-a841-0fa6be74c316 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:39.914265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ea019-0172-4d93-aa7c-9b2307705eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:39.915876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ea019-0172-4d93-aa7c-9b2307705eaa None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:39.916198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ea019-0172-4d93-aa7c-9b2307705eaa None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:39.916483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ea019-0172-4d93-aa7c-9b2307705eaa None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:39.917221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ea019-0172-4d93-aa7c-9b2307705eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:39.917532 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ea019-0172-4d93-aa7c-9b2307705eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:39.918938 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ea019-0172-4d93-aa7c-9b2307705eaa None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:39.934491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 365/733] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:39.936351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4be03-fd69-44a6-80e1-747c23c9e751 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:39.936809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4be03-fd69-44a6-80e1-747c23c9e751 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:39.937193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4be03-fd69-44a6-80e1-747c23c9e751 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:39.940218 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 366/734] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:39.941170 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4be03-fd69-44a6-80e1-747c23c9e751 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:40.031342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bdd57-3d38-44af-a841-0fa6be74c316 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:40.070238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4be03-fd69-44a6-80e1-747c23c9e751 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:40.071464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:40.080039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bd661-7e4b-4b23-ad99-5ea6c4173469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:40.080848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bd661-7e4b-4b23-ad99-5ea6c4173469 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:40.081235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bd661-7e4b-4b23-ad99-5ea6c4173469 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:40.081610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bd661-7e4b-4b23-ad99-5ea6c4173469 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:40.119046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca46ec-9f59-4efc-9ca7-68f87b8729d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:40.120693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca46ec-9f59-4efc-9ca7-68f87b8729d4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:40.121078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca46ec-9f59-4efc-9ca7-68f87b8729d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:40.121492 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca46ec-9f59-4efc-9ca7-68f87b8729d4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:40.132371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca46ec-9f59-4efc-9ca7-68f87b8729d4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:40.133949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca46ec-9f59-4efc-9ca7-68f87b8729d4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-acca46ec-9f59-4efc-9ca7-68f87b8729d4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:40.153175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:40.154600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:40.156437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:40.156437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:40.156437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:40.156437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:40.156437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:40.156437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:40.156437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:40.156437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:40.156437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 03:36:40.156437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:40.157848 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 367/735] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:40.159534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9147-12fb-4d75-9602-3bc945a75b4b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:40.159918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9147-12fb-4d75-9602-3bc945a75b4b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:40.160770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9147-12fb-4d75-9602-3bc945a75b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:40.176877 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bd661-7e4b-4b23-ad99-5ea6c4173469 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:40.179349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bd661-7e4b-4b23-ad99-5ea6c4173469 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:40.181810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-741c9147-12fb-4d75-9602-3bc945a75b4b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:40.184021 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-741c9147-12fb-4d75-9602-3bc945a75b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:40.193836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:40.266855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 368/736] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:40.268211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c9147-12fb-4d75-9602-3bc945a75b4b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:40.301931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 369/737] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:40.304434 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bd661-7e4b-4b23-ad99-5ea6c4173469 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:40.347116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:40.405465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c9147-12fb-4d75-9602-3bc945a75b4b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:40.427653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bd661-7e4b-4b23-ad99-5ea6c4173469 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:40.443360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d76118-18cd-413e-ab1e-04b0aa11e20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:40.444090 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d76118-18cd-413e-ab1e-04b0aa11e20c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:40.444447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d76118-18cd-413e-ab1e-04b0aa11e20c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:40.444695 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d76118-18cd-413e-ab1e-04b0aa11e20c None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:40.445501 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d76118-18cd-413e-ab1e-04b0aa11e20c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:40.445781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d76118-18cd-413e-ab1e-04b0aa11e20c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:40.447245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d76118-18cd-413e-ab1e-04b0aa11e20c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:40.461244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:40.466539 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0481972-f949-49af-b6a9-c231feb93460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:40.467211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0481972-f949-49af-b6a9-c231feb93460 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:40.467499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0481972-f949-49af-b6a9-c231feb93460 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:40.467779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0481972-f949-49af-b6a9-c231feb93460 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:40.468143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 369/738] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Jul 12 03:36:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:40.468640 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0481972-f949-49af-b6a9-c231feb93460 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:40.469988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0481972-f949-49af-b6a9-c231feb93460 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:40.470589 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d76118-18cd-413e-ab1e-04b0aa11e20c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:40.486701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 370/739] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:36:40 2022] POST /identity/v3/services => generated 240 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:36:40.488144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892e8ff-abff-4a47-97e0-478aa9507c4c None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:40.488437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892e8ff-abff-4a47-97e0-478aa9507c4c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:40.488727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892e8ff-abff-4a47-97e0-478aa9507c4c None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:40.490252 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 371/740] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:40.495208 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892e8ff-abff-4a47-97e0-478aa9507c4c None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:40.606667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d76118-18cd-413e-ab1e-04b0aa11e20c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:40.622663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:40.653828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892e8ff-abff-4a47-97e0-478aa9507c4c None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:40.658165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd8206-0814-4543-b242-bc6483be0a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:40.660936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd8206-0814-4543-b242-bc6483be0a29 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:40.661227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd8206-0814-4543-b242-bc6483be0a29 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:40.661508 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd8206-0814-4543-b242-bc6483be0a29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:40.673498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd8206-0814-4543-b242-bc6483be0a29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:40.674966 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd8206-0814-4543-b242-bc6483be0a29 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f8dd8206-0814-4543-b242-bc6483be0a29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:40.695452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:40.696467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:40.697764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:40.697764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:40.697764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:40.697764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:40.697764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 370/741] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:40.698574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbceef-2cb8-4c47-bd70-76be8c6a921f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:40.698851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbceef-2cb8-4c47-bd70-76be8c6a921f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:40.699193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbceef-2cb8-4c47-bd70-76be8c6a921f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:40.703281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3c399-66a3-4ffe-a8af-ea83f5f568a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:40.703934 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3c399-66a3-4ffe-a8af-ea83f5f568a0 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:40.704485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3c399-66a3-4ffe-a8af-ea83f5f568a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:40.704668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3c399-66a3-4ffe-a8af-ea83f5f568a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:40.726042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-eafbceef-2cb8-4c47-bd70-76be8c6a921f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:40.727232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-eafbceef-2cb8-4c47-bd70-76be8c6a921f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:40.781646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:40.791403 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3c399-66a3-4ffe-a8af-ea83f5f568a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:40.793135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3c399-66a3-4ffe-a8af-ea83f5f568a0 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:40.802921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 371/742] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:40 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:40.809052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbceef-2cb8-4c47-bd70-76be8c6a921f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:40.888565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:40.906752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 372/743] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:40 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 03:36:40.912573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e6dbe-183d-4806-8ff8-b8f15302a791 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:40.913089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e6dbe-183d-4806-8ff8-b8f15302a791 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:40.913196 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e6dbe-183d-4806-8ff8-b8f15302a791 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:40.918521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 373/744] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:40.918521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e6dbe-183d-4806-8ff8-b8f15302a791 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:40.923405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbceef-2cb8-4c47-bd70-76be8c6a921f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:40.965172 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e68f63-7c5a-42f4-b710-aaea142b9a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:40.965881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e68f63-7c5a-42f4-b710-aaea142b9a3b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:40.966505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e68f63-7c5a-42f4-b710-aaea142b9a3b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:40.966799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e68f63-7c5a-42f4-b710-aaea142b9a3b None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:40.967870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e68f63-7c5a-42f4-b710-aaea142b9a3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:40.968239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e68f63-7c5a-42f4-b710-aaea142b9a3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:40.969880 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e68f63-7c5a-42f4-b710-aaea142b9a3b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:40.986651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 372/745] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:40.986651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e68f63-7c5a-42f4-b710-aaea142b9a3b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:41.032487 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:41.032728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e6dbe-183d-4806-8ff8-b8f15302a791 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:41.068597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1db93f-a619-4a27-b8cb-d315590eeea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:41.069411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1db93f-a619-4a27-b8cb-d315590eeea9 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:41.069690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1db93f-a619-4a27-b8cb-d315590eeea9 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:41.069999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1db93f-a619-4a27-b8cb-d315590eeea9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:41.081432 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1db93f-a619-4a27-b8cb-d315590eeea9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-7b1db93f-a619-4a27-b8cb-d315590eeea9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:41.089028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:41.089985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:41.090805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 374/746] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:41.091488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08524650-7a69-4ee6-be4f-d7315028f052 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:41.092078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08524650-7a69-4ee6-be4f-d7315028f052 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:41.092078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08524650-7a69-4ee6-be4f-d7315028f052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:41.103233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e68f63-7c5a-42f4-b710-aaea142b9a3b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:41.109227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-08524650-7a69-4ee6-be4f-d7315028f052 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:41.110461 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-08524650-7a69-4ee6-be4f-d7315028f052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:41.140276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893b76b-294a-408e-881e-720d59d58be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:41.141502 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893b76b-294a-408e-881e-720d59d58be8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:41.141993 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893b76b-294a-408e-881e-720d59d58be8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:41.142419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893b76b-294a-408e-881e-720d59d58be8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:41.153955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893b76b-294a-408e-881e-720d59d58be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:41.154228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:41.155418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893b76b-294a-408e-881e-720d59d58be8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a893b76b-294a-408e-881e-720d59d58be8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:41.172480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:41.173367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:41.174231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:41.174231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:41.174231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:41.174231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:41.174231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 373/747] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:41.174511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893b76b-294a-408e-881e-720d59d58be8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:41.186765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 375/748] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:41 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 03:36:41.188043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08524650-7a69-4ee6-be4f-d7315028f052 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:41.290775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:41.298826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893b76b-294a-408e-881e-720d59d58be8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:41.302013 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08524650-7a69-4ee6-be4f-d7315028f052 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:41.333819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f115b0b-2427-49b6-8de8-e492f290d8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:41.334801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f115b0b-2427-49b6-8de8-e492f290d8c7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:41.335175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f115b0b-2427-49b6-8de8-e492f290d8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:41.335550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f115b0b-2427-49b6-8de8-e492f290d8c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:41.336473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c81aa-6c10-4365-843d-012b4a607d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:41.337142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c81aa-6c10-4365-843d-012b4a607d07 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:41.337487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c81aa-6c10-4365-843d-012b4a607d07 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:41.337768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c81aa-6c10-4365-843d-012b4a607d07 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:41.338545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c81aa-6c10-4365-843d-012b4a607d07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:41.338855 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c81aa-6c10-4365-843d-012b4a607d07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:41.340268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c81aa-6c10-4365-843d-012b4a607d07 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:41.343328 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f115b0b-2427-49b6-8de8-e492f290d8c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:41.345073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f115b0b-2427-49b6-8de8-e492f290d8c7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:41.349059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 376/749] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:41.349834 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c81aa-6c10-4365-843d-012b4a607d07 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-7f115b0b-2427-49b6-8de8-e492f290d8c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:41.360292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:41.361205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:41.362075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:41.362075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:41.362075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:41.362075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:41.362075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:41.362075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:41.362075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:41.362075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:41.364347 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 374/750] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:41.365429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f115b0b-2427-49b6-8de8-e492f290d8c7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:41.393653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:41.461475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c81aa-6c10-4365-843d-012b4a607d07 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:41.468400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f115b0b-2427-49b6-8de8-e492f290d8c7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:41.498017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac34ee9-9b1a-4543-bda2-fc699df290d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:41.498668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac34ee9-9b1a-4543-bda2-fc699df290d8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:41.498962 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac34ee9-9b1a-4543-bda2-fc699df290d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:41.499223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac34ee9-9b1a-4543-bda2-fc699df290d8 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:41.499957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac34ee9-9b1a-4543-bda2-fc699df290d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:41.500349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac34ee9-9b1a-4543-bda2-fc699df290d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:41.501777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac34ee9-9b1a-4543-bda2-fc699df290d8 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:41.506481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce1883-a4e8-4686-a732-18242f3bdd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:41.507182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce1883-a4e8-4686-a732-18242f3bdd66 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:41.507525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce1883-a4e8-4686-a732-18242f3bdd66 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:41.507722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce1883-a4e8-4686-a732-18242f3bdd66 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:41.508472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce1883-a4e8-4686-a732-18242f3bdd66 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:41.509883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce1883-a4e8-4686-a732-18242f3bdd66 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:41.515380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 377/751] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:41.517084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac34ee9-9b1a-4543-bda2-fc699df290d8 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:41.532219 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 375/752] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:36:41 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:36:41.533080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce1883-a4e8-4686-a732-18242f3bdd66 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:41.560731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:41.704592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac34ee9-9b1a-4543-bda2-fc699df290d8 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:41.708423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:41.711866 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce1883-a4e8-4686-a732-18242f3bdd66 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:41.760449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac65540-3c59-47c2-b8a5-06087e68b4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:41.761259 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac65540-3c59-47c2-b8a5-06087e68b4bf None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:41.761538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac65540-3c59-47c2-b8a5-06087e68b4bf None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:41.761819 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac65540-3c59-47c2-b8a5-06087e68b4bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:41.770855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bcc7e-6536-40f8-9e69-502c2b48bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:41.770855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bcc7e-6536-40f8-9e69-502c2b48bb1a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:41.770855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bcc7e-6536-40f8-9e69-502c2b48bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:41.770855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bcc7e-6536-40f8-9e69-502c2b48bb1a None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:41.772300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bcc7e-6536-40f8-9e69-502c2b48bb1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:41.772858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bcc7e-6536-40f8-9e69-502c2b48bb1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:41.774670 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bcc7e-6536-40f8-9e69-502c2b48bb1a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:41.776553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac65540-3c59-47c2-b8a5-06087e68b4bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:41.778081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac65540-3c59-47c2-b8a5-06087e68b4bf None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:41.795153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 376/753] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:41.796436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bcc7e-6536-40f8-9e69-502c2b48bb1a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-7ac65540-3c59-47c2-b8a5-06087e68b4bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:41.924659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:41.925613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:41.926500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:41.926500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:41.926500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:41.926500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:41.930868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 378/754] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:41.931721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac65540-3c59-47c2-b8a5-06087e68b4bf None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:41.938122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:41.988200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bcc7e-6536-40f8-9e69-502c2b48bb1a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:42.052825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f306c-d9a6-476f-90bc-63f539f6982b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:42.055552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f306c-d9a6-476f-90bc-63f539f6982b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:42.056255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f306c-d9a6-476f-90bc-63f539f6982b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:42.057390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f306c-d9a6-476f-90bc-63f539f6982b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:42.078954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f306c-d9a6-476f-90bc-63f539f6982b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:42.080971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f306c-d9a6-476f-90bc-63f539f6982b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:42.102459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac65540-3c59-47c2-b8a5-06087e68b4bf None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-902f306c-d9a6-476f-90bc-63f539f6982b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:42.110247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:42.111201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:42.112124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:42.112124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:42.112124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:42.112124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:42.112124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:42.112124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:42.112124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:42.112124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:42.119465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 377/755] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:42.120610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f306c-d9a6-476f-90bc-63f539f6982b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:42.169933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6225231-5cc4-4e66-a429-760ed06faba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:42.170614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6225231-5cc4-4e66-a429-760ed06faba5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:42.170887 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6225231-5cc4-4e66-a429-760ed06faba5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:42.176025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6225231-5cc4-4e66-a429-760ed06faba5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:42.176025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6225231-5cc4-4e66-a429-760ed06faba5 None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=2b01c3b5c3384d4ca0999e9156c3a9cb) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:42.176025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6225231-5cc4-4e66-a429-760ed06faba5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:42.176025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6225231-5cc4-4e66-a429-760ed06faba5 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:42.190046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 379/756] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:41 2022] GET /identity/v3/role_assignments?user.id=2b01c3b5c3384d4ca0999e9156c3a9cb&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:42.191040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6225231-5cc4-4e66-a429-760ed06faba5 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:42.233117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:42.345001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f306c-d9a6-476f-90bc-63f539f6982b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:42.375838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:42.389869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ecaca-3afc-4ea5-859a-71e8989a6166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:42.390555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ecaca-3afc-4ea5-859a-71e8989a6166 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:42.390936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ecaca-3afc-4ea5-859a-71e8989a6166 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:42.391361 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ecaca-3afc-4ea5-859a-71e8989a6166 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:42.392261 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ecaca-3afc-4ea5-859a-71e8989a6166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:42.392575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ecaca-3afc-4ea5-859a-71e8989a6166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:42.393987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ecaca-3afc-4ea5-859a-71e8989a6166 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:42.408906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6225231-5cc4-4e66-a429-760ed06faba5 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:42.410238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 378/757] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:42.417243 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ecaca-3afc-4ea5-859a-71e8989a6166 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:42.451734 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e9964-b050-45af-a0a1-54afb7afceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:42.452449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e9964-b050-45af-a0a1-54afb7afceed None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:42.452736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e9964-b050-45af-a0a1-54afb7afceed None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:42.453015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e9964-b050-45af-a0a1-54afb7afceed None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:42.453884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e9964-b050-45af-a0a1-54afb7afceed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:42.454195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e9964-b050-45af-a0a1-54afb7afceed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:42.455652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e9964-b050-45af-a0a1-54afb7afceed None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:42.479060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 380/758] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:42.485224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e9964-b050-45af-a0a1-54afb7afceed None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:42.585322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:42.610218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e9964-b050-45af-a0a1-54afb7afceed None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:42.626203 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ecaca-3afc-4ea5-859a-71e8989a6166 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:42.652842 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932c7f0-4051-475d-9f23-65d4596623da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:42.653512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932c7f0-4051-475d-9f23-65d4596623da None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:42.653800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932c7f0-4051-475d-9f23-65d4596623da None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:42.654083 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932c7f0-4051-475d-9f23-65d4596623da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:42.654914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932c7f0-4051-475d-9f23-65d4596623da None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=0026b8f11158492b90da3395cc2c43c6) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:42.655228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932c7f0-4051-475d-9f23-65d4596623da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:42.657369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932c7f0-4051-475d-9f23-65d4596623da None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:42.668249 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 381/759] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:42 2022] GET /identity/v3/role_assignments?user.id=0026b8f11158492b90da3395cc2c43c6&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:42.678119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe84d3-27b1-40a3-b8d2-83abb8f1ffe4 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:42.678551 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe84d3-27b1-40a3-b8d2-83abb8f1ffe4 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:42.678926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe84d3-27b1-40a3-b8d2-83abb8f1ffe4 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:42.681803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 382/760] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:42.687341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d403cce-38fb-4620-b4d1-1b0d66195b69 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:42.687913 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d403cce-38fb-4620-b4d1-1b0d66195b69 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:42.688328 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d403cce-38fb-4620-b4d1-1b0d66195b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:42.706695 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345bf2a-d1c2-42f7-9c56-9cd2fbcf29cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:42.707864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345bf2a-d1c2-42f7-9c56-9cd2fbcf29cd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:42.708379 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345bf2a-d1c2-42f7-9c56-9cd2fbcf29cd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:42.708800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345bf2a-d1c2-42f7-9c56-9cd2fbcf29cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:42.723094 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8d403cce-38fb-4620-b4d1-1b0d66195b69 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:42.724175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345bf2a-d1c2-42f7-9c56-9cd2fbcf29cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:42.726269 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345bf2a-d1c2-42f7-9c56-9cd2fbcf29cd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:42.729601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8d403cce-38fb-4620-b4d1-1b0d66195b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6345bf2a-d1c2-42f7-9c56-9cd2fbcf29cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:42.758701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:42.759680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:42.760691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:42.760691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:42.760691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:42.760691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:42.760691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:42.760691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:42.760691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:42.760691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:42.760691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:36:42.760691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:42.765281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 379/761] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 12 03:36:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:42.772373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345bf2a-d1c2-42f7-9c56-9cd2fbcf29cd None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:42.859949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:42.865190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 383/762] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:36:42.889755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d403cce-38fb-4620-b4d1-1b0d66195b69 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:42.918800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345bf2a-d1c2-42f7-9c56-9cd2fbcf29cd None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:42.954820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedb5eb-964b-43f7-8755-5d5fc5403422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:42.955870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedb5eb-964b-43f7-8755-5d5fc5403422 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:42.956383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedb5eb-964b-43f7-8755-5d5fc5403422 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:42.956888 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedb5eb-964b-43f7-8755-5d5fc5403422 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:42.957943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bedb5eb-964b-43f7-8755-5d5fc5403422 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:42.958459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bedb5eb-964b-43f7-8755-5d5fc5403422 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:42.960168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bedb5eb-964b-43f7-8755-5d5fc5403422 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:42.979484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 380/763] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 12 03:36:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:42.985202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedb5eb-964b-43f7-8755-5d5fc5403422 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:43.078873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:43.090029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedb5eb-964b-43f7-8755-5d5fc5403422 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:43.099861 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:43.129875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b86707-0704-4260-a252-55bd2d519c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:43.130736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b86707-0704-4260-a252-55bd2d519c6f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:43.131012 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b86707-0704-4260-a252-55bd2d519c6f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:43.131283 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b86707-0704-4260-a252-55bd2d519c6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:43.149126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b86707-0704-4260-a252-55bd2d519c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:43.150572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b86707-0704-4260-a252-55bd2d519c6f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:43.153596 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d403cce-38fb-4620-b4d1-1b0d66195b69 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-34b86707-0704-4260-a252-55bd2d519c6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:43.180131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:43.181104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:43.182035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:43.182035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:43.182035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:43.182035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:43.183909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 381/764] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:43.191355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b86707-0704-4260-a252-55bd2d519c6f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:43.210386 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7cb5a-5b99-483a-9174-23b6e1d0c45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:43.211889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7cb5a-5b99-483a-9174-23b6e1d0c45d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:43.212342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7cb5a-5b99-483a-9174-23b6e1d0c45d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:43.212762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7cb5a-5b99-483a-9174-23b6e1d0c45d None admin] PATH_INFO: `/v3/services/network` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:43.213597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7cb5a-5b99-483a-9174-23b6e1d0c45d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:43.215135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7cb5a-5b99-483a-9174-23b6e1d0c45d None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-47b7cb5a-5b99-483a-9174-23b6e1d0c45d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:43.228625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:43.229865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 03:36:43.230935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:43.233838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 384/765] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:42 2022] GET /identity/v3/services/network => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:43.240379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7cb5a-5b99-483a-9174-23b6e1d0c45d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:43.354043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7cb5a-5b99-483a-9174-23b6e1d0c45d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:43.393310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b86707-0704-4260-a252-55bd2d519c6f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:43.415039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:43.423606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229d46a-88a4-4c3f-9100-5c09d921b27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:43.424335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229d46a-88a4-4c3f-9100-5c09d921b27a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:43.424620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229d46a-88a4-4c3f-9100-5c09d921b27a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:43.424895 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229d46a-88a4-4c3f-9100-5c09d921b27a None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:43.425621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229d46a-88a4-4c3f-9100-5c09d921b27a None admin] RBAC: Adding query filter params (name=network) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:43.425927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229d46a-88a4-4c3f-9100-5c09d921b27a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:43.427331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229d46a-88a4-4c3f-9100-5c09d921b27a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:43.443704 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 385/766] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:36:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:43.445454 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229d46a-88a4-4c3f-9100-5c09d921b27a None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:43.467220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071346e7-d87a-408f-ab4b-87795ec540fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:43.467895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071346e7-d87a-408f-ab4b-87795ec540fb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:43.468199 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071346e7-d87a-408f-ab4b-87795ec540fb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:43.468478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071346e7-d87a-408f-ab4b-87795ec540fb None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:43.469282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071346e7-d87a-408f-ab4b-87795ec540fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:43.469590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071346e7-d87a-408f-ab4b-87795ec540fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:43.471022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071346e7-d87a-408f-ab4b-87795ec540fb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:43.506731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 382/767] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:43.513094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071346e7-d87a-408f-ab4b-87795ec540fb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:43.572383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:43.573555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229d46a-88a4-4c3f-9100-5c09d921b27a None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:43.608608 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39e9a5-b03f-4656-96d5-60ed16e4fedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:43.609229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39e9a5-b03f-4656-96d5-60ed16e4fedc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:43.609517 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39e9a5-b03f-4656-96d5-60ed16e4fedc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:43.609798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39e9a5-b03f-4656-96d5-60ed16e4fedc None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:43.610514 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39e9a5-b03f-4656-96d5-60ed16e4fedc None admin] RBAC: Adding query filter params (type=network) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:43.610821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39e9a5-b03f-4656-96d5-60ed16e4fedc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:43.612230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39e9a5-b03f-4656-96d5-60ed16e4fedc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:43.622259 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 386/768] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:36:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:43.623458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39e9a5-b03f-4656-96d5-60ed16e4fedc None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:43.684450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:43.728523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39e9a5-b03f-4656-96d5-60ed16e4fedc None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:43.751810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071346e7-d87a-408f-ab4b-87795ec540fb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:43.764916 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65473804-c4eb-434f-a6cb-1aa33a3a0042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:43.765616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65473804-c4eb-434f-a6cb-1aa33a3a0042 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:43.765904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65473804-c4eb-434f-a6cb-1aa33a3a0042 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:43.766183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65473804-c4eb-434f-a6cb-1aa33a3a0042 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:43.766970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65473804-c4eb-434f-a6cb-1aa33a3a0042 None admin] RBAC: Adding query filter params (interface=public, service_id=1ca57c0a1f2d49de87abdfc3009cdea3, region_id=RegionOne) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:43.767285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65473804-c4eb-434f-a6cb-1aa33a3a0042 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:43.768815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65473804-c4eb-434f-a6cb-1aa33a3a0042 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:43.781213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 387/769] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 12 03:36:43 2022] GET /identity/v3/endpoints?service_id=1ca57c0a1f2d49de87abdfc3009cdea3&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:43.782506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65473804-c4eb-434f-a6cb-1aa33a3a0042 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:43.825650 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca094622-3644-43e9-a4f7-ca1ddc7cb633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:43.826462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca094622-3644-43e9-a4f7-ca1ddc7cb633 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:43.826742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca094622-3644-43e9-a4f7-ca1ddc7cb633 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:43.827014 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca094622-3644-43e9-a4f7-ca1ddc7cb633 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:43.840054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca094622-3644-43e9-a4f7-ca1ddc7cb633 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:43.841543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca094622-3644-43e9-a4f7-ca1ddc7cb633 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-ca094622-3644-43e9-a4f7-ca1ddc7cb633 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:43.877586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:43.878535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:43.879446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:43.879446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:43.879446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:43.879446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:43.879446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:43.879446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:43.879446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:43.879446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:43.879446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 383/770] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:43.886540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca094622-3644-43e9-a4f7-ca1ddc7cb633 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:43.894319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65473804-c4eb-434f-a6cb-1aa33a3a0042 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:43.924479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:43.932133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bac70-a290-4495-8898-3413940ad47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:43.932821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bac70-a290-4495-8898-3413940ad47e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:43.933093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bac70-a290-4495-8898-3413940ad47e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:43.933358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bac70-a290-4495-8898-3413940ad47e None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:43.934060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bac70-a290-4495-8898-3413940ad47e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:43.935485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bac70-a290-4495-8898-3413940ad47e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:43.945774 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 388/771] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 12 03:36:43 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:44.003899 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:44.027132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa68bcd-472a-4fcb-a04d-650af3be0879 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:44.027465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa68bcd-472a-4fcb-a04d-650af3be0879 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:44.027661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa68bcd-472a-4fcb-a04d-650af3be0879 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:44.029114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 389/772] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:44.035133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4fc9d-30c4-4aa3-b76d-f606cd4dc6c1 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:44.035911 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca094622-3644-43e9-a4f7-ca1ddc7cb633 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:44.036123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4fc9d-30c4-4aa3-b76d-f606cd4dc6c1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:44.036629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4fc9d-30c4-4aa3-b76d-f606cd4dc6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:44.073247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f1c4fc9d-30c4-4aa3-b76d-f606cd4dc6c1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:44.074905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f1c4fc9d-30c4-4aa3-b76d-f606cd4dc6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:44.075600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd2084-af25-404d-a0a6-2c51f36b99d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:44.076339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd2084-af25-404d-a0a6-2c51f36b99d7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:44.076619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd2084-af25-404d-a0a6-2c51f36b99d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:44.076892 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd2084-af25-404d-a0a6-2c51f36b99d7 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:44.077694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd2084-af25-404d-a0a6-2c51f36b99d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:44.078180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd2084-af25-404d-a0a6-2c51f36b99d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:44.081381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd2084-af25-404d-a0a6-2c51f36b99d7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:44.094393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 384/773] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:44.100258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd2084-af25-404d-a0a6-2c51f36b99d7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:44.198035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 390/774] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:36:44.220789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4fc9d-30c4-4aa3-b76d-f606cd4dc6c1 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:44.223723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd2084-af25-404d-a0a6-2c51f36b99d7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:44.260950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955ba03-f1c7-4ad2-bb7c-3b53d39c1fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:44.262207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955ba03-f1c7-4ad2-bb7c-3b53d39c1fce None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:44.262494 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955ba03-f1c7-4ad2-bb7c-3b53d39c1fce None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:44.262866 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955ba03-f1c7-4ad2-bb7c-3b53d39c1fce None admin] PATH_INFO: `/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/7138318e53ff4b7885942d49b19eb0a5/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:44.265251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955ba03-f1c7-4ad2-bb7c-3b53d39c1fce None admin] RBAC: Authorizing `identity:create_grant(project_id=48b17053daa84e4fa94840777cb1bee6, user_id=7138318e53ff4b7885942d49b19eb0a5, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:44.266863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955ba03-f1c7-4ad2-bb7c-3b53d39c1fce None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:44.278324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:44.290457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 385/775] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:36:44 2022] PUT /identity/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/7138318e53ff4b7885942d49b19eb0a5/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:36:44.346780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4fc9d-30c4-4aa3-b76d-f606cd4dc6c1 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:44.369084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:44.391089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aef12-a611-4719-a82c-0ba14acd9776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:44.391957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aef12-a611-4719-a82c-0ba14acd9776 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:44.392260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aef12-a611-4719-a82c-0ba14acd9776 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:44.392574 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aef12-a611-4719-a82c-0ba14acd9776 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:44.393264 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54aef12-a611-4719-a82c-0ba14acd9776 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:44.394757 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54aef12-a611-4719-a82c-0ba14acd9776 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-c54aef12-a611-4719-a82c-0ba14acd9776 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:44.408573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:44.409570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:44.411091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 391/776] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Jul 12 03:36:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:44.416433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955ba03-f1c7-4ad2-bb7c-3b53d39c1fce None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:44.566408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955ba03-f1c7-4ad2-bb7c-3b53d39c1fce None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:44.606682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dc745-4509-4fbf-a53f-ae26f4bf4a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:44.607395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dc745-4509-4fbf-a53f-ae26f4bf4a42 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:44.607676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dc745-4509-4fbf-a53f-ae26f4bf4a42 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:44.607944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dc745-4509-4fbf-a53f-ae26f4bf4a42 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:44.608689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97dc745-4509-4fbf-a53f-ae26f4bf4a42 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:44.608996 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97dc745-4509-4fbf-a53f-ae26f4bf4a42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:44.610525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97dc745-4509-4fbf-a53f-ae26f4bf4a42 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:44.616158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 386/777] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Jul 12 03:36:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:44.621975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aef12-a611-4719-a82c-0ba14acd9776 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:44.678898 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591b92f-fe42-4890-abf8-535d159b25f1 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:44.679246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591b92f-fe42-4890-abf8-535d159b25f1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:44.679456 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591b92f-fe42-4890-abf8-535d159b25f1 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:44.681151 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 387/778] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:44.686889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418d99c-539b-4a98-8124-8ecc2c9ad464 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:44.687365 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418d99c-539b-4a98-8124-8ecc2c9ad464 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:44.687648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418d99c-539b-4a98-8124-8ecc2c9ad464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:44.711318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4418d99c-539b-4a98-8124-8ecc2c9ad464 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:44.712665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4418d99c-539b-4a98-8124-8ecc2c9ad464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:44.735657 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aef12-a611-4719-a82c-0ba14acd9776 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:44.798558 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b88f0d-4abb-4d3a-a09a-d29b3b0d0ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:44.799464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b88f0d-4abb-4d3a-a09a-d29b3b0d0ba0 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:44.799756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b88f0d-4abb-4d3a-a09a-d29b3b0d0ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:44.800049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b88f0d-4abb-4d3a-a09a-d29b3b0d0ba0 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:44.801224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b88f0d-4abb-4d3a-a09a-d29b3b0d0ba0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:44.801677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b88f0d-4abb-4d3a-a09a-d29b3b0d0ba0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:44.803066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b88f0d-4abb-4d3a-a09a-d29b3b0d0ba0 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:44.805224 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:44.810973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 392/779] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Jul 12 03:36:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:44.816673 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b88f0d-4abb-4d3a-a09a-d29b3b0d0ba0 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:44.822810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 388/780] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:36:44.872386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418d99c-539b-4a98-8124-8ecc2c9ad464 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:44.894696 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:45.005293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b88f0d-4abb-4d3a-a09a-d29b3b0d0ba0 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:45.034212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418d99c-539b-4a98-8124-8ecc2c9ad464 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:45.054132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a472921-915f-4ad8-802d-1aba2c04a437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:45.054794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a472921-915f-4ad8-802d-1aba2c04a437 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:45.055078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a472921-915f-4ad8-802d-1aba2c04a437 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:45.055354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a472921-915f-4ad8-802d-1aba2c04a437 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:45.056177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a472921-915f-4ad8-802d-1aba2c04a437 None admin] RBAC: Adding query filter params (interface=public, service_id=f663c014c33543dc95563c4e7e0855e6, region_id=RegionOne) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:45.056489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a472921-915f-4ad8-802d-1aba2c04a437 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:45.057872 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a472921-915f-4ad8-802d-1aba2c04a437 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:45.063318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 393/781] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 12 03:36:44 2022] GET /identity/v3/endpoints?service_id=f663c014c33543dc95563c4e7e0855e6&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:45.064778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fe9d-a453-4ba3-8db2-5485e643b015 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:45.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fe9d-a453-4ba3-8db2-5485e643b015 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:45.065345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1fe9d-a453-4ba3-8db2-5485e643b015 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:45.070934 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 394/782] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:45.071742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1fe9d-a453-4ba3-8db2-5485e643b015 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:45.072794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453694ea-f7fb-4a7c-b2c4-6adf05aa858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:45.073601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453694ea-f7fb-4a7c-b2c4-6adf05aa858b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:45.073877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453694ea-f7fb-4a7c-b2c4-6adf05aa858b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:45.074151 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453694ea-f7fb-4a7c-b2c4-6adf05aa858b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:45.085460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453694ea-f7fb-4a7c-b2c4-6adf05aa858b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-453694ea-f7fb-4a7c-b2c4-6adf05aa858b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:45.093828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:45.095477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:45.097217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:45.097217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:45.097217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:45.097217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:45.097217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:45.097217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:45.097217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:45.097217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:45.097217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:45.097217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:45.097217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:45.097217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:45.104024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 389/783] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:45.104024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a56d9-33b2-478d-8925-bed001c25c00 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:45.104024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a56d9-33b2-478d-8925-bed001c25c00 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:45.104024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a56d9-33b2-478d-8925-bed001c25c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:45.125054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-378a56d9-33b2-478d-8925-bed001c25c00 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:45.127261 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-378a56d9-33b2-478d-8925-bed001c25c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:45.192571 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:45.196089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1fe9d-a453-4ba3-8db2-5485e643b015 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:45.210619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:45.222813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 390/784] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:36:45.223988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a56d9-33b2-478d-8925-bed001c25c00 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:45.254604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e086f-ba70-44f6-a542-ae10b8c13775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:45.255693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e086f-ba70-44f6-a542-ae10b8c13775 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:45.255979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e086f-ba70-44f6-a542-ae10b8c13775 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:45.256290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e086f-ba70-44f6-a542-ae10b8c13775 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:45.256979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e086f-ba70-44f6-a542-ae10b8c13775 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:45.258369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e086f-ba70-44f6-a542-ae10b8c13775 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:45.272047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 395/785] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 12 03:36:45 2022] GET /identity/v3/services => generated 772 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:45.272047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e086f-ba70-44f6-a542-ae10b8c13775 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:45.397561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e086f-ba70-44f6-a542-ae10b8c13775 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:45.439589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c356f-a495-41a8-88f2-266eccf9d16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:45.440985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c356f-a495-41a8-88f2-266eccf9d16e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:45.441400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c356f-a495-41a8-88f2-266eccf9d16e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:45.441681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c356f-a495-41a8-88f2-266eccf9d16e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:45.443189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a56d9-33b2-478d-8925-bed001c25c00 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:45.456292 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c356f-a495-41a8-88f2-266eccf9d16e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-819c356f-a495-41a8-88f2-266eccf9d16e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:45.466234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:45.467181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:45.468039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:45.468039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:45.468039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:45.468039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:45.468039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:45.468039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:45.468039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:45.468039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:45.468039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:45.468039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:45.468039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:45.468039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:45.469269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 396/786] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:45.474552 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c356f-a495-41a8-88f2-266eccf9d16e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:45.489745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be6c70-c986-40bb-adba-6a9f1639a5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:45.490914 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be6c70-c986-40bb-adba-6a9f1639a5d1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:45.491201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be6c70-c986-40bb-adba-6a9f1639a5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:45.491483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be6c70-c986-40bb-adba-6a9f1639a5d1 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:45.492944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be6c70-c986-40bb-adba-6a9f1639a5d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:45.493266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be6c70-c986-40bb-adba-6a9f1639a5d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:45.494952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13be6c70-c986-40bb-adba-6a9f1639a5d1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:45.507498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 391/787] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:45.513072 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be6c70-c986-40bb-adba-6a9f1639a5d1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:45.525012 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:45.580351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:45.606081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c356f-a495-41a8-88f2-266eccf9d16e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:45.661234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90042dec-5214-448f-9466-5ac684c784fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:45.661852 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90042dec-5214-448f-9466-5ac684c784fa None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:45.662087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90042dec-5214-448f-9466-5ac684c784fa None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:45.662366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90042dec-5214-448f-9466-5ac684c784fa None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:45.663343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90042dec-5214-448f-9466-5ac684c784fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:45.663656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90042dec-5214-448f-9466-5ac684c784fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:45.665697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be6c70-c986-40bb-adba-6a9f1639a5d1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:45.669550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90042dec-5214-448f-9466-5ac684c784fa None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:45.686861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 397/788] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:45.692778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90042dec-5214-448f-9466-5ac684c784fa None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:45.748780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ab5ec-4b4a-4369-ad50-ab4bce4548a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:45.749786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ab5ec-4b4a-4369-ad50-ab4bce4548a4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:45.750080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ab5ec-4b4a-4369-ad50-ab4bce4548a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:45.750352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ab5ec-4b4a-4369-ad50-ab4bce4548a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:45.768477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ab5ec-4b4a-4369-ad50-ab4bce4548a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:45.770472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ab5ec-4b4a-4369-ad50-ab4bce4548a4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a69ab5ec-4b4a-4369-ad50-ab4bce4548a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:45.800666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:45.801641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:45.802538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:45.802538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:45.802538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:45.802538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:45.804930 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:45.810863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 392/789] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:45.812922 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90042dec-5214-448f-9466-5ac684c784fa None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:45.816895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ab5ec-4b4a-4369-ad50-ab4bce4548a4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:45.849385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f6838-9373-4c93-8459-5dbc31b76131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:45.850206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f6838-9373-4c93-8459-5dbc31b76131 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:45.850493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f6838-9373-4c93-8459-5dbc31b76131 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:45.850779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f6838-9373-4c93-8459-5dbc31b76131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:45.862520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f6838-9373-4c93-8459-5dbc31b76131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:45.864035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f6838-9373-4c93-8459-5dbc31b76131 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-172f6838-9373-4c93-8459-5dbc31b76131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:45.882578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:45.883634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:45.884561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:45.884561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:45.884561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:45.884561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:45.884561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 398/790] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:45.892318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f6838-9373-4c93-8459-5dbc31b76131 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:45.986339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ab5ec-4b4a-4369-ad50-ab4bce4548a4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:45.989807 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:46.021774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d98b46-9a45-42c6-be02-05a0135ad670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:46.022823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98b46-9a45-42c6-be02-05a0135ad670 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:46.023345 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98b46-9a45-42c6-be02-05a0135ad670 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:46.023834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98b46-9a45-42c6-be02-05a0135ad670 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:46.024886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d98b46-9a45-42c6-be02-05a0135ad670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:46.025403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d98b46-9a45-42c6-be02-05a0135ad670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:46.027076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d98b46-9a45-42c6-be02-05a0135ad670 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:46.040117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 393/791] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:46.049089 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d98b46-9a45-42c6-be02-05a0135ad670 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:46.071055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f6838-9373-4c93-8459-5dbc31b76131 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:46.132722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a2b8f-212a-458b-b572-b31d765f2e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:46.133400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a2b8f-212a-458b-b572-b31d765f2e02 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:46.133683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a2b8f-212a-458b-b572-b31d765f2e02 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:46.133953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a2b8f-212a-458b-b572-b31d765f2e02 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:46.134675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a2b8f-212a-458b-b572-b31d765f2e02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:46.134977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a2b8f-212a-458b-b572-b31d765f2e02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:46.137265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a2b8f-212a-458b-b572-b31d765f2e02 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:46.167588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d98b46-9a45-42c6-be02-05a0135ad670 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:46.170918 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 399/792] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:46.189072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a2b8f-212a-458b-b572-b31d765f2e02 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:46.209763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de722cdc-55e7-43ab-8986-7db1de034c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:46.211318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de722cdc-55e7-43ab-8986-7db1de034c9f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:46.211823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de722cdc-55e7-43ab-8986-7db1de034c9f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:46.212418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de722cdc-55e7-43ab-8986-7db1de034c9f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:46.217946 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:46.223807 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de722cdc-55e7-43ab-8986-7db1de034c9f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:46.226597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de722cdc-55e7-43ab-8986-7db1de034c9f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-de722cdc-55e7-43ab-8986-7db1de034c9f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:46.245085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:46.246181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:46.247222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:46.247222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:46.247222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:46.247222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:46.247222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:46.247222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:46.247222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:46.247222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:46.247222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 03:36:46.247222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:46.250236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 394/793] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 12 03:36:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:46.255591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de722cdc-55e7-43ab-8986-7db1de034c9f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:46.280390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:46.361502 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de722cdc-55e7-43ab-8986-7db1de034c9f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:46.398029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040794a4-e720-4a09-a84d-15b8cda7cd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:46.399209 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040794a4-e720-4a09-a84d-15b8cda7cd29 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:46.399704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040794a4-e720-4a09-a84d-15b8cda7cd29 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:46.400204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040794a4-e720-4a09-a84d-15b8cda7cd29 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:46.401372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040794a4-e720-4a09-a84d-15b8cda7cd29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:46.401890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040794a4-e720-4a09-a84d-15b8cda7cd29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:46.403646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040794a4-e720-4a09-a84d-15b8cda7cd29 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:46.407187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a2b8f-212a-458b-b572-b31d765f2e02 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:46.423078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 395/794] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Jul 12 03:36:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:46.429100 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040794a4-e720-4a09-a84d-15b8cda7cd29 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:46.456748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90e7cf-3171-451a-a876-1d6d3f699fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:46.457571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90e7cf-3171-451a-a876-1d6d3f699fd0 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:46.457857 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90e7cf-3171-451a-a876-1d6d3f699fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:46.458137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90e7cf-3171-451a-a876-1d6d3f699fd0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:46.466058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90e7cf-3171-451a-a876-1d6d3f699fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:46.467465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90e7cf-3171-451a-a876-1d6d3f699fd0 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-2f90e7cf-3171-451a-a876-1d6d3f699fd0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:46.499896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:46.500900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:46.501830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:46.501830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:46.501830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:46.501830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:46.501830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:46.501830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:46.501830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:46.501830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:46.501830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 03:36:46.501830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:46.508251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 400/795] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:46.513559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90e7cf-3171-451a-a876-1d6d3f699fd0 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:46.524374 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:46.543755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040794a4-e720-4a09-a84d-15b8cda7cd29 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:46.583056 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536290c-bf3e-4db5-809c-736ace4e6e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:46.584028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536290c-bf3e-4db5-809c-736ace4e6e86 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:46.584398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536290c-bf3e-4db5-809c-736ace4e6e86 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:46.584737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536290c-bf3e-4db5-809c-736ace4e6e86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:46.596261 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536290c-bf3e-4db5-809c-736ace4e6e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:46.597905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536290c-bf3e-4db5-809c-736ace4e6e86 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-1536290c-bf3e-4db5-809c-736ace4e6e86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:46.617356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:46.618306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:46.619354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:46.619354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:46.619354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:46.619354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:46.619354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 396/796] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:46.620390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:46.625493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb1974-0a5e-47e8-988b-4e2bb2d4acce None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:46.625814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb1974-0a5e-47e8-988b-4e2bb2d4acce None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:46.626122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb1974-0a5e-47e8-988b-4e2bb2d4acce None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:46.627911 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 397/797] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:46.629525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb1974-0a5e-47e8-988b-4e2bb2d4acce None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:46.673512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90e7cf-3171-451a-a876-1d6d3f699fd0 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:46.717453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58436d71-6931-403c-bda0-97659430dcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:46.718691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58436d71-6931-403c-bda0-97659430dcce None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:46.719241 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58436d71-6931-403c-bda0-97659430dcce None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:46.719884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58436d71-6931-403c-bda0-97659430dcce None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:46.721680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58436d71-6931-403c-bda0-97659430dcce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:46.723051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58436d71-6931-403c-bda0-97659430dcce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:46.724879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58436d71-6931-403c-bda0-97659430dcce None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:46.750067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 401/798] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Jul 12 03:36:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:46.752041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084ab41-6395-4256-a054-9ddbb1b98dff None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:46.752668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084ab41-6395-4256-a054-9ddbb1b98dff None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:46.753237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084ab41-6395-4256-a054-9ddbb1b98dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:46.763079 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb1974-0a5e-47e8-988b-4e2bb2d4acce None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:46.773145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d084ab41-6395-4256-a054-9ddbb1b98dff None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:46.774824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d084ab41-6395-4256-a054-9ddbb1b98dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:46.790922 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:46.799541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca879884-22d2-4bd3-9f51-ea6bbbec59b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:46.800246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca879884-22d2-4bd3-9f51-ea6bbbec59b8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:46.800531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca879884-22d2-4bd3-9f51-ea6bbbec59b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:46.800800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca879884-22d2-4bd3-9f51-ea6bbbec59b8 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:46.801532 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca879884-22d2-4bd3-9f51-ea6bbbec59b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:46.801831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca879884-22d2-4bd3-9f51-ea6bbbec59b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:46.803225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca879884-22d2-4bd3-9f51-ea6bbbec59b8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:46.820153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 398/799] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:46.821034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca879884-22d2-4bd3-9f51-ea6bbbec59b8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:46.853300 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 402/800] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:36:46.854500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084ab41-6395-4256-a054-9ddbb1b98dff None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:46.912407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:46.931248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca879884-22d2-4bd3-9f51-ea6bbbec59b8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:46.966525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084ab41-6395-4256-a054-9ddbb1b98dff None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:46.970828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa6672e-1ab8-454e-bf2c-f44f9f1b7023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:46.971575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6672e-1ab8-454e-bf2c-f44f9f1b7023 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:46.971855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6672e-1ab8-454e-bf2c-f44f9f1b7023 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:46.972159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6672e-1ab8-454e-bf2c-f44f9f1b7023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:46.983858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6672e-1ab8-454e-bf2c-f44f9f1b7023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:46.985349 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa6672e-1ab8-454e-bf2c-f44f9f1b7023 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:47.000769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e38f4e-50b7-4365-bcdb-b0c5a86f618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:47.001572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e38f4e-50b7-4365-bcdb-b0c5a86f618e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.001855 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e38f4e-50b7-4365-bcdb-b0c5a86f618e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.002131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e38f4e-50b7-4365-bcdb-b0c5a86f618e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-8fa6672e-1ab8-454e-bf2c-f44f9f1b7023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:47.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:47.003513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:47.004433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:47.004433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:47.004433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:47.004433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:47.004433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 399/801] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:47.005169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51637ea7-d03f-4c90-a695-ee78bf846048 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.005421 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51637ea7-d03f-4c90-a695-ee78bf846048 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.005689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51637ea7-d03f-4c90-a695-ee78bf846048 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.010946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 400/802] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:47.010946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51637ea7-d03f-4c90-a695-ee78bf846048 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:47.017098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e38f4e-50b7-4365-bcdb-b0c5a86f618e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:47.018601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e38f4e-50b7-4365-bcdb-b0c5a86f618e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-c6e38f4e-50b7-4365-bcdb-b0c5a86f618e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:47.032390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:47.033290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:47.034203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:47.034203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:47.034203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:47.034203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:47.034203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:47.034203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:47.034203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:47.034203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:47.038622 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 403/803] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:47.039251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e38f4e-50b7-4365-bcdb-b0c5a86f618e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:47.057389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:47.125811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51637ea7-d03f-4c90-a695-ee78bf846048 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:47.142475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e38f4e-50b7-4365-bcdb-b0c5a86f618e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:47.159723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:47.161469 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a1504-8d95-4c19-8734-7761129ae320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:47.162236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a1504-8d95-4c19-8734-7761129ae320 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.162520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a1504-8d95-4c19-8734-7761129ae320 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.162804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a1504-8d95-4c19-8734-7761129ae320 None admin] PATH_INFO: `/v3/services/network` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.163461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a1504-8d95-4c19-8734-7761129ae320 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:47.165389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a1504-8d95-4c19-8734-7761129ae320 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:47.178075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb994a-e870-4b73-a8f5-86e25be2a79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:47.178743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb994a-e870-4b73-a8f5-86e25be2a79e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.179027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb994a-e870-4b73-a8f5-86e25be2a79e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-313a1504-8d95-4c19-8734-7761129ae320 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:47.179245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:47.180543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 03:36:47.181377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:47.182117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb994a-e870-4b73-a8f5-86e25be2a79e None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.182117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb994a-e870-4b73-a8f5-86e25be2a79e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:47.182117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb994a-e870-4b73-a8f5-86e25be2a79e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:47.182117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb994a-e870-4b73-a8f5-86e25be2a79e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:47.185405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 401/804] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:47 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:47.186714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317f45c-8117-4c07-96a5-8b8b0fe8694f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.187079 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317f45c-8117-4c07-96a5-8b8b0fe8694f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.187360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317f45c-8117-4c07-96a5-8b8b0fe8694f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.194810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 404/805] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:47.195551 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb994a-e870-4b73-a8f5-86e25be2a79e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:47.207263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d317f45c-8117-4c07-96a5-8b8b0fe8694f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:47.208491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d317f45c-8117-4c07-96a5-8b8b0fe8694f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:47.288101 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 402/806] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:36:47.288101 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317f45c-8117-4c07-96a5-8b8b0fe8694f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:47.298890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb994a-e870-4b73-a8f5-86e25be2a79e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:47.325252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:47.334831 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4469c19-38f3-4235-8803-177fd4d50de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:47.336040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4469c19-38f3-4235-8803-177fd4d50de0 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.336332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4469c19-38f3-4235-8803-177fd4d50de0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.336897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4469c19-38f3-4235-8803-177fd4d50de0 None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.337717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4469c19-38f3-4235-8803-177fd4d50de0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:47.338033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4469c19-38f3-4235-8803-177fd4d50de0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:47.339505 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4469c19-38f3-4235-8803-177fd4d50de0 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:47.356368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 405/807] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:47.356368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4469c19-38f3-4235-8803-177fd4d50de0 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:47.391022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:47.401215 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317f45c-8117-4c07-96a5-8b8b0fe8694f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:47.436824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ee8b6-066c-443e-8c45-9da2a9c722c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:47.437535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ee8b6-066c-443e-8c45-9da2a9c722c1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.437824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ee8b6-066c-443e-8c45-9da2a9c722c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.438103 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ee8b6-066c-443e-8c45-9da2a9c722c1 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.438821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ee8b6-066c-443e-8c45-9da2a9c722c1 None admin] RBAC: Adding query filter params (name=network) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:47.439122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ee8b6-066c-443e-8c45-9da2a9c722c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:47.440570 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ee8b6-066c-443e-8c45-9da2a9c722c1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:47.445215 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 403/808] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:36:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:47.449826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ee8b6-066c-443e-8c45-9da2a9c722c1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:47.465231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4469c19-38f3-4235-8803-177fd4d50de0 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:47.500562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2c90-e53d-472d-969a-4c9bbf7b4ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:47.501607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2c90-e53d-472d-969a-4c9bbf7b4ef4 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.502194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2c90-e53d-472d-969a-4c9bbf7b4ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.502537 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2c90-e53d-472d-969a-4c9bbf7b4ef4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.509976 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d2c90-e53d-472d-969a-4c9bbf7b4ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:47.511794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d2c90-e53d-472d-969a-4c9bbf7b4ef4 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-145d2c90-e53d-472d-969a-4c9bbf7b4ef4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:47.526312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:47.527275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:47.528317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:47.528317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:47.528317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:47.528317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:47.528317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:47.528317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:47.528317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:47.528317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:47.528317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 406/809] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:47.529784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2c90-e53d-472d-969a-4c9bbf7b4ef4 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:47.556734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:47.560844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ee8b6-066c-443e-8c45-9da2a9c722c1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:47.599276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9b87f-a5b3-4e60-ab13-cede03203d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:47.600127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9b87f-a5b3-4e60-ab13-cede03203d04 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.600414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9b87f-a5b3-4e60-ab13-cede03203d04 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.600690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9b87f-a5b3-4e60-ab13-cede03203d04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.611404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9b87f-a5b3-4e60-ab13-cede03203d04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-22e9b87f-a5b3-4e60-ab13-cede03203d04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:47.620891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:47.622787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 404/810] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:47.623502 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9b87f-a5b3-4e60-ab13-cede03203d04 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:47.639352 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:47.645019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2c90-e53d-472d-969a-4c9bbf7b4ef4 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:47.678625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474c3e1-fe06-45d6-8bd1-da93d757bccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:47.679511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474c3e1-fe06-45d6-8bd1-da93d757bccd None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.679792 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474c3e1-fe06-45d6-8bd1-da93d757bccd None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.680095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474c3e1-fe06-45d6-8bd1-da93d757bccd None admin] PATH_INFO: `/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/2b01c3b5c3384d4ca0999e9156c3a9cb/roles/f653b35aa347415e953be7ead75d9fd0` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.694003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474c3e1-fe06-45d6-8bd1-da93d757bccd None admin] RBAC: Authorizing `identity:create_grant(project_id=48b17053daa84e4fa94840777cb1bee6, user_id=2b01c3b5c3384d4ca0999e9156c3a9cb, role_id=f653b35aa347415e953be7ead75d9fd0)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:47.695518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474c3e1-fe06-45d6-8bd1-da93d757bccd None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:47.707815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 407/811] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:36:47 2022] PUT /identity/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/2b01c3b5c3384d4ca0999e9156c3a9cb/roles/f653b35aa347415e953be7ead75d9fd0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:36:47.708643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474c3e1-fe06-45d6-8bd1-da93d757bccd None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:47.729086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9b87f-a5b3-4e60-ab13-cede03203d04 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:47.766835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d9130-bf7b-403a-b933-cb7e4f2a604f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:47.767816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d9130-bf7b-403a-b933-cb7e4f2a604f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.768634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d9130-bf7b-403a-b933-cb7e4f2a604f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.768928 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d9130-bf7b-403a-b933-cb7e4f2a604f None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.769660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d9130-bf7b-403a-b933-cb7e4f2a604f None admin] RBAC: Adding query filter params (type=network) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:47.769976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d9130-bf7b-403a-b933-cb7e4f2a604f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:47.771422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d9130-bf7b-403a-b933-cb7e4f2a604f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:47.782065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 405/812] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:36:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:47.782065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d9130-bf7b-403a-b933-cb7e4f2a604f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:47.808541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:47.835294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474c3e1-fe06-45d6-8bd1-da93d757bccd None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:47.873783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be547f-fb49-4d4f-8489-59e2035aafc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:47.874595 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be547f-fb49-4d4f-8489-59e2035aafc2 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.874891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be547f-fb49-4d4f-8489-59e2035aafc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.875171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be547f-fb49-4d4f-8489-59e2035aafc2 None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.876034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be547f-fb49-4d4f-8489-59e2035aafc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:47.876360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be547f-fb49-4d4f-8489-59e2035aafc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:47.877808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be547f-fb49-4d4f-8489-59e2035aafc2 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:47.890695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 408/813] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:47.892068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a42767-b28f-49ab-a3f8-33e0ccf563ff None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.892201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d9130-bf7b-403a-b933-cb7e4f2a604f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:47.892611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a42767-b28f-49ab-a3f8-33e0ccf563ff None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.892611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a42767-b28f-49ab-a3f8-33e0ccf563ff None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.893836 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 409/814] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:47.894517 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a42767-b28f-49ab-a3f8-33e0ccf563ff None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:47.905287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:47.930732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218dfc40-14fc-4928-935b-3ea3b0780d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:47.931425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218dfc40-14fc-4928-935b-3ea3b0780d06 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:47.931677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218dfc40-14fc-4928-935b-3ea3b0780d06 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:47.931954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218dfc40-14fc-4928-935b-3ea3b0780d06 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:47.932768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218dfc40-14fc-4928-935b-3ea3b0780d06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:47.933069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218dfc40-14fc-4928-935b-3ea3b0780d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:47.934471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218dfc40-14fc-4928-935b-3ea3b0780d06 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:47.944040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 406/815] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:47.944864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218dfc40-14fc-4928-935b-3ea3b0780d06 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:48.014315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a42767-b28f-49ab-a3f8-33e0ccf563ff None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:48.037752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:48.051523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556be09-0e09-4252-8b10-9e10ab50ba4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:48.052310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556be09-0e09-4252-8b10-9e10ab50ba4a None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:48.052597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556be09-0e09-4252-8b10-9e10ab50ba4a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:48.052879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556be09-0e09-4252-8b10-9e10ab50ba4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:48.053716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556be09-0e09-4252-8b10-9e10ab50ba4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:48.055122 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556be09-0e09-4252-8b10-9e10ab50ba4a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:48.055871 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218dfc40-14fc-4928-935b-3ea3b0780d06 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:48.079111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 410/816] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 12 03:36:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:36:48.080357 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f7362-db5a-4bf3-af5d-f4c1137db94d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:48.080649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f7362-db5a-4bf3-af5d-f4c1137db94d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:48.080929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f7362-db5a-4bf3-af5d-f4c1137db94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:48.102658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-6f1f7362-db5a-4bf3-af5d-f4c1137db94d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:48.103918 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6f1f7362-db5a-4bf3-af5d-f4c1137db94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:48.123359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9b256-c167-4a64-b09c-9e5e649bc0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:48.124370 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9b256-c167-4a64-b09c-9e5e649bc0d7 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:48.124682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9b256-c167-4a64-b09c-9e5e649bc0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:48.124921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9b256-c167-4a64-b09c-9e5e649bc0d7 None admin] PATH_INFO: `/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/0026b8f11158492b90da3395cc2c43c6/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:48.127365 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9b256-c167-4a64-b09c-9e5e649bc0d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=48b17053daa84e4fa94840777cb1bee6, user_id=0026b8f11158492b90da3395cc2c43c6, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:48.128935 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9b256-c167-4a64-b09c-9e5e649bc0d7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:48.152035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 407/817] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:36:47 2022] PUT /identity/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/0026b8f11158492b90da3395cc2c43c6/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:36:48.152035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9b256-c167-4a64-b09c-9e5e649bc0d7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:48.248424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:48.273484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 411/818] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:36:48.280579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9b256-c167-4a64-b09c-9e5e649bc0d7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:48.298845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f7362-db5a-4bf3-af5d-f4c1137db94d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:48.318259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81085c8-ad5a-4e9f-a3dd-46572fda846f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:48.319176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81085c8-ad5a-4e9f-a3dd-46572fda846f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:48.319525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81085c8-ad5a-4e9f-a3dd-46572fda846f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:48.319866 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81085c8-ad5a-4e9f-a3dd-46572fda846f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:48.331165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:48.336438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81085c8-ad5a-4e9f-a3dd-46572fda846f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:48.338143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81085c8-ad5a-4e9f-a3dd-46572fda846f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e81085c8-ad5a-4e9f-a3dd-46572fda846f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:48.361200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:48.362351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:48.363400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:48.363400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:48.363400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:48.363400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:48.363400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 408/819] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:48.370805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81085c8-ad5a-4e9f-a3dd-46572fda846f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:48.440022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f7362-db5a-4bf3-af5d-f4c1137db94d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:48.483210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845665ee-21d6-47af-8736-e38a03704867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:48.484353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845665ee-21d6-47af-8736-e38a03704867 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:48.484795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845665ee-21d6-47af-8736-e38a03704867 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:48.485209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845665ee-21d6-47af-8736-e38a03704867 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:48.486379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845665ee-21d6-47af-8736-e38a03704867 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:48.487980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845665ee-21d6-47af-8736-e38a03704867 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:48.500175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81085c8-ad5a-4e9f-a3dd-46572fda846f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-845665ee-21d6-47af-8736-e38a03704867 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:48.503268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:48.504460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 03:36:48.505462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:48.507990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 412/820] 10.222.0.20 () {56 vars in 1129 bytes} [Tue Jul 12 03:36:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:48.523751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845665ee-21d6-47af-8736-e38a03704867 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:48.539126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d96987-3323-4a73-994a-bd006e0fc7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:48.539793 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d96987-3323-4a73-994a-bd006e0fc7a8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:48.540104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d96987-3323-4a73-994a-bd006e0fc7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:48.540386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d96987-3323-4a73-994a-bd006e0fc7a8 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:48.541545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d96987-3323-4a73-994a-bd006e0fc7a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:48.541861 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d96987-3323-4a73-994a-bd006e0fc7a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:48.543486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d96987-3323-4a73-994a-bd006e0fc7a8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:48.559384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 409/821] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:48.564507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d96987-3323-4a73-994a-bd006e0fc7a8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:48.580872 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:48.625523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:48.626297 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845665ee-21d6-47af-8736-e38a03704867 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:48.663305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e2f6d-ca59-436e-8678-4b3833a296da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:48.664116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e2f6d-ca59-436e-8678-4b3833a296da None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:48.664525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e2f6d-ca59-436e-8678-4b3833a296da None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:48.664874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e2f6d-ca59-436e-8678-4b3833a296da None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:48.665751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e2f6d-ca59-436e-8678-4b3833a296da None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:48.666132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e2f6d-ca59-436e-8678-4b3833a296da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:48.667581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e2f6d-ca59-436e-8678-4b3833a296da None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:48.677396 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 413/822] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Jul 12 03:36:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:48.678525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e2f6d-ca59-436e-8678-4b3833a296da None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:48.685313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d96987-3323-4a73-994a-bd006e0fc7a8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:48.720286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920d20-4b15-4f0f-90b6-328314190009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:48.721387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920d20-4b15-4f0f-90b6-328314190009 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:48.721806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920d20-4b15-4f0f-90b6-328314190009 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:48.722215 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920d20-4b15-4f0f-90b6-328314190009 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:48.729925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920d20-4b15-4f0f-90b6-328314190009 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:48.731618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920d20-4b15-4f0f-90b6-328314190009 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-68920d20-4b15-4f0f-90b6-328314190009 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:48.746620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:48.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:48.748929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:48.748929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:48.748929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:48.748929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:48.748929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:48.748929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:48.748929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:48.748929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:48.748929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:36:48.748929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:48.751317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 410/823] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 12 03:36:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:48.756795 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920d20-4b15-4f0f-90b6-328314190009 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:48.794298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e2f6d-ca59-436e-8678-4b3833a296da None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:48.830102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2d0d8-8b64-40b8-b6d8-c4c2c4aa1f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:48.830649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2d0d8-8b64-40b8-b6d8-c4c2c4aa1f90 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:48.831007 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2d0d8-8b64-40b8-b6d8-c4c2c4aa1f90 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:48.831355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2d0d8-8b64-40b8-b6d8-c4c2c4aa1f90 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:48.832176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2d0d8-8b64-40b8-b6d8-c4c2c4aa1f90 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:48.832572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2d0d8-8b64-40b8-b6d8-c4c2c4aa1f90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:48.834048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2d0d8-8b64-40b8-b6d8-c4c2c4aa1f90 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:48.844359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 414/824] 10.222.0.20 () {56 vars in 1124 bytes} [Tue Jul 12 03:36:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:48.844359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:48.845872 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2d0d8-8b64-40b8-b6d8-c4c2c4aa1f90 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:48.871524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920d20-4b15-4f0f-90b6-328314190009 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:48.891038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:48.917751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1709a5-eb7a-462f-9bd6-17ba3511730b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:48.919147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1709a5-eb7a-462f-9bd6-17ba3511730b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:48.919564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1709a5-eb7a-462f-9bd6-17ba3511730b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:48.919991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1709a5-eb7a-462f-9bd6-17ba3511730b None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:48.921125 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1709a5-eb7a-462f-9bd6-17ba3511730b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:48.921561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1709a5-eb7a-462f-9bd6-17ba3511730b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:48.923153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1709a5-eb7a-462f-9bd6-17ba3511730b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:48.946293 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 411/825] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 12 03:36:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:48.949353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1709a5-eb7a-462f-9bd6-17ba3511730b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:48.975018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2d0d8-8b64-40b8-b6d8-c4c2c4aa1f90 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:49.011651 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953af3f-3bf1-4985-aff1-72ddeaa23dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:49.012527 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953af3f-3bf1-4985-aff1-72ddeaa23dd7 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:49.012914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953af3f-3bf1-4985-aff1-72ddeaa23dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:49.013288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953af3f-3bf1-4985-aff1-72ddeaa23dd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:49.014147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953af3f-3bf1-4985-aff1-72ddeaa23dd7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:49.015842 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953af3f-3bf1-4985-aff1-72ddeaa23dd7 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:49.040042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 415/826] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 12 03:36:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:36:49.074956 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1709a5-eb7a-462f-9bd6-17ba3511730b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:49.119490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78feb1-8af0-43b4-aba4-49dcf36f5c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:49.120206 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78feb1-8af0-43b4-aba4-49dcf36f5c68 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:49.120578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78feb1-8af0-43b4-aba4-49dcf36f5c68 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:49.120887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78feb1-8af0-43b4-aba4-49dcf36f5c68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:49.158776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:49.196168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78feb1-8af0-43b4-aba4-49dcf36f5c68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:49.197701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78feb1-8af0-43b4-aba4-49dcf36f5c68 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:49.300686 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 412/827] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:36:49.306342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953af3f-3bf1-4985-aff1-72ddeaa23dd7 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:49.353293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:49.416060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953af3f-3bf1-4985-aff1-72ddeaa23dd7 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:49.451287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c9fb8-bcca-49a0-9eda-9828f42704b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:49.452585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c9fb8-bcca-49a0-9eda-9828f42704b8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:49.453072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c9fb8-bcca-49a0-9eda-9828f42704b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:49.453545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c9fb8-bcca-49a0-9eda-9828f42704b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:49.465655 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c9fb8-bcca-49a0-9eda-9828f42704b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:49.467384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c9fb8-bcca-49a0-9eda-9828f42704b8 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-735c9fb8-bcca-49a0-9eda-9828f42704b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:49.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:49.486380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:49.487288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:49.487288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:49.487288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:49.487288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:49.489601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 416/828] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:49.496932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78feb1-8af0-43b4-aba4-49dcf36f5c68 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:49.561412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:49.599623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78feb1-8af0-43b4-aba4-49dcf36f5c68 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:49.633693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521cabb-d341-49b9-9407-4ea53f06143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:49.634378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521cabb-d341-49b9-9407-4ea53f06143a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:49.634662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521cabb-d341-49b9-9407-4ea53f06143a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:49.634934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521cabb-d341-49b9-9407-4ea53f06143a None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:49.635650 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521cabb-d341-49b9-9407-4ea53f06143a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:49.635954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521cabb-d341-49b9-9407-4ea53f06143a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:49.637763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521cabb-d341-49b9-9407-4ea53f06143a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:49.649881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 413/829] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:49.655384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c9fb8-bcca-49a0-9eda-9828f42704b8 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:49.751055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:49.759422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c9fb8-bcca-49a0-9eda-9828f42704b8 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:49.793882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645618d6-4fd0-4145-9bf3-4049c69a6caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:49.794673 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645618d6-4fd0-4145-9bf3-4049c69a6caa None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:49.795220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645618d6-4fd0-4145-9bf3-4049c69a6caa None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:49.795220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645618d6-4fd0-4145-9bf3-4049c69a6caa None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:49.802242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645618d6-4fd0-4145-9bf3-4049c69a6caa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:49.803685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645618d6-4fd0-4145-9bf3-4049c69a6caa None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:49.818506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8baa1-37ae-4178-bb21-7d24288dd51b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:49.818798 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8baa1-37ae-4178-bb21-7d24288dd51b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:49.819060 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8baa1-37ae-4178-bb21-7d24288dd51b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:49.820687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 414/830] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-645618d6-4fd0-4145-9bf3-4049c69a6caa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:49.821422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:49.822383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:49.823255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:49.823255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:49.823255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:49.823255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:49.823255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:49.823255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:49.823255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:49.823255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:49.823255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 417/831] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:49.828399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4f501-414e-4779-a2b1-a1edbefd0402 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:49.828652 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4f501-414e-4779-a2b1-a1edbefd0402 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:49.829187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4f501-414e-4779-a2b1-a1edbefd0402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:49.831146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645618d6-4fd0-4145-9bf3-4049c69a6caa None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:49.849131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a5a4f501-414e-4779-a2b1-a1edbefd0402 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:49.850632 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a5a4f501-414e-4779-a2b1-a1edbefd0402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:49.937481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 415/832] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:36:49.949437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645618d6-4fd0-4145-9bf3-4049c69a6caa None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:49.960240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4f501-414e-4779-a2b1-a1edbefd0402 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:50.005423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a463c-1694-4fd4-8e84-a1bfd7cc5eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:50.006280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a463c-1694-4fd4-8e84-a1bfd7cc5eee None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:50.006569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a463c-1694-4fd4-8e84-a1bfd7cc5eee None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:50.006847 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a463c-1694-4fd4-8e84-a1bfd7cc5eee None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:50.007899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a463c-1694-4fd4-8e84-a1bfd7cc5eee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:50.008242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a463c-1694-4fd4-8e84-a1bfd7cc5eee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:50.010479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a463c-1694-4fd4-8e84-a1bfd7cc5eee None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:50.019357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:50.028044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 418/833] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:50.037023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a463c-1694-4fd4-8e84-a1bfd7cc5eee None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:50.060420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:50.067658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4f501-414e-4779-a2b1-a1edbefd0402 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:50.111865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f960b-9d98-4421-99f7-c30c3cd6b592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:50.113257 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f960b-9d98-4421-99f7-c30c3cd6b592 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:50.114040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f960b-9d98-4421-99f7-c30c3cd6b592 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:50.114735 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f960b-9d98-4421-99f7-c30c3cd6b592 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:50.128890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f960b-9d98-4421-99f7-c30c3cd6b592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-420f960b-9d98-4421-99f7-c30c3cd6b592 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:50.142154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:50.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:50.143919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:50.143919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:50.143919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:50.143919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:50.143919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:50.143919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:50.143919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:50.143919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:50.143919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:50.143919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:50.143919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:36:50.143919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:50.152037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 416/834] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:36:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:50.154451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f960b-9d98-4421-99f7-c30c3cd6b592 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:50.165131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a463c-1694-4fd4-8e84-a1bfd7cc5eee None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:50.200689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5a7a5-c5f5-4352-b7ff-bc35a0152d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:50.201623 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5a7a5-c5f5-4352-b7ff-bc35a0152d09 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:50.202128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5a7a5-c5f5-4352-b7ff-bc35a0152d09 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:50.202549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5a7a5-c5f5-4352-b7ff-bc35a0152d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:50.203519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5a7a5-c5f5-4352-b7ff-bc35a0152d09 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=7138318e53ff4b7885942d49b19eb0a5) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:50.204040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5a7a5-c5f5-4352-b7ff-bc35a0152d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:50.205659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5a7a5-c5f5-4352-b7ff-bc35a0152d09 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:50.212296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 419/835] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:50 2022] GET /identity/v3/role_assignments?user.id=7138318e53ff4b7885942d49b19eb0a5&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:50.274469 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1d1b4-aa05-4623-aa70-76a424d5f1a0 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:50.274770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1d1b4-aa05-4623-aa70-76a424d5f1a0 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:50.274958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1d1b4-aa05-4623-aa70-76a424d5f1a0 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:50.276585 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f960b-9d98-4421-99f7-c30c3cd6b592 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:50.280682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 420/836] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:50.282528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410071c-189d-46ac-8168-4a0d365d0d5f None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:50.282821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410071c-189d-46ac-8168-4a0d365d0d5f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:50.283102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410071c-189d-46ac-8168-4a0d365d0d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:50.303231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-7410071c-189d-46ac-8168-4a0d365d0d5f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:50.304504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7410071c-189d-46ac-8168-4a0d365d0d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:50.316535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:50.333225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e4ccd-aa38-42cc-868d-21c4c197d406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:50.334453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e4ccd-aa38-42cc-868d-21c4c197d406 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:50.335127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e4ccd-aa38-42cc-868d-21c4c197d406 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:50.335550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e4ccd-aa38-42cc-868d-21c4c197d406 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:50.337174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e4ccd-aa38-42cc-868d-21c4c197d406 None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:50.337642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e4ccd-aa38-42cc-868d-21c4c197d406 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:50.340188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e4ccd-aa38-42cc-868d-21c4c197d406 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:50.351852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 417/837] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:36:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:50.357417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e4ccd-aa38-42cc-868d-21c4c197d406 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:50.392150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 421/838] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:36:50.407592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410071c-189d-46ac-8168-4a0d365d0d5f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:50.417510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:50.486708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e4ccd-aa38-42cc-868d-21c4c197d406 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:50.527187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b7f92-3a33-46e2-a21a-cd95d6f4981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:50.528308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b7f92-3a33-46e2-a21a-cd95d6f4981c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:50.528592 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b7f92-3a33-46e2-a21a-cd95d6f4981c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:50.528867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b7f92-3a33-46e2-a21a-cd95d6f4981c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:50.542059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410071c-189d-46ac-8168-4a0d365d0d5f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:50.542462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b7f92-3a33-46e2-a21a-cd95d6f4981c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:50.543936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b7f92-3a33-46e2-a21a-cd95d6f4981c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-b61b7f92-3a33-46e2-a21a-cd95d6f4981c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:50.564990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:50.565955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:50.566881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:50.566881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:50.566881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:50.566881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:50.566881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 418/839] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:50.571904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b7f92-3a33-46e2-a21a-cd95d6f4981c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:50.584737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55290f47-6772-4a43-8f56-ce0c768c2eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:50.586343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55290f47-6772-4a43-8f56-ce0c768c2eba None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:50.586831 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55290f47-6772-4a43-8f56-ce0c768c2eba None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:50.587653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55290f47-6772-4a43-8f56-ce0c768c2eba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:50.604519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55290f47-6772-4a43-8f56-ce0c768c2eba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:50.608718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-55290f47-6772-4a43-8f56-ce0c768c2eba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:50.618237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:50.619197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:50.620048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:50.620048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:50.620048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:50.620048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:50.620048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:50.620048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:50.620048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:50.620048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:50.620048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:50.620048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:50.620048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:36:50.620048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:50.623236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 422/840] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:50.629253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55290f47-6772-4a43-8f56-ce0c768c2eba None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:50.688576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b7f92-3a33-46e2-a21a-cd95d6f4981c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:50.698932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:50.727361 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091b747-e56e-4414-83f5-44be188b57b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:50.728476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091b747-e56e-4414-83f5-44be188b57b9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:50.729047 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091b747-e56e-4414-83f5-44be188b57b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:50.729579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091b747-e56e-4414-83f5-44be188b57b9 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:50.730544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091b747-e56e-4414-83f5-44be188b57b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:50.731150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091b747-e56e-4414-83f5-44be188b57b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:50.732965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091b747-e56e-4414-83f5-44be188b57b9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:50.735891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55290f47-6772-4a43-8f56-ce0c768c2eba None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:50.746881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 419/841] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:50.752143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091b747-e56e-4414-83f5-44be188b57b9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:50.786744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16e417-ecea-4016-8e89-7d541fd1caae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:50.787362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e417-ecea-4016-8e89-7d541fd1caae None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:50.787647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e417-ecea-4016-8e89-7d541fd1caae None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:50.787924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16e417-ecea-4016-8e89-7d541fd1caae None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:50.788703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16e417-ecea-4016-8e89-7d541fd1caae None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:50.789011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16e417-ecea-4016-8e89-7d541fd1caae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:50.790392 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16e417-ecea-4016-8e89-7d541fd1caae None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:50.799785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 423/842] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:36:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:50.808306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16e417-ecea-4016-8e89-7d541fd1caae None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:50.848239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:50.859239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091b747-e56e-4414-83f5-44be188b57b9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:50.896632 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424e7bc-c3c5-46b0-8f0a-8144f435f2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:50.897549 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424e7bc-c3c5-46b0-8f0a-8144f435f2d2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:50.897903 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424e7bc-c3c5-46b0-8f0a-8144f435f2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:50.898186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424e7bc-c3c5-46b0-8f0a-8144f435f2d2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:50.906756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424e7bc-c3c5-46b0-8f0a-8144f435f2d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:50.908258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424e7bc-c3c5-46b0-8f0a-8144f435f2d2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-7424e7bc-c3c5-46b0-8f0a-8144f435f2d2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:50.924263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:50.925235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:50.926819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:50.926819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:50.926819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:50.926819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:50.926819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:50.926819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:50.926819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:50.926819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:50.926819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 03:36:50.926819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:50.926819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 420/843] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 12 03:36:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:50.931312 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424e7bc-c3c5-46b0-8f0a-8144f435f2d2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:50.941314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16e417-ecea-4016-8e89-7d541fd1caae None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:50.979490 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:50.989337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a8558-79cb-4e36-b774-1756bbfb9a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:50.990163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a8558-79cb-4e36-b774-1756bbfb9a66 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:50.990448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a8558-79cb-4e36-b774-1756bbfb9a66 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:50.990727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a8558-79cb-4e36-b774-1756bbfb9a66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:51.003977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a8558-79cb-4e36-b774-1756bbfb9a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:51.005606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a8558-79cb-4e36-b774-1756bbfb9a66 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-ad0a8558-79cb-4e36-b774-1756bbfb9a66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:51.025894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:51.026866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:51.027766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:51.027766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:51.027766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:51.027766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:51.032148 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 424/844] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:51.033693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a8558-79cb-4e36-b774-1756bbfb9a66 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:51.146397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a8558-79cb-4e36-b774-1756bbfb9a66 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:51.190883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b777a-7dc7-43e1-bcdc-f4f0910bbacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:51.192094 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b777a-7dc7-43e1-bcdc-f4f0910bbacc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:51.192563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b777a-7dc7-43e1-bcdc-f4f0910bbacc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:51.193358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b777a-7dc7-43e1-bcdc-f4f0910bbacc None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:51.194474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b777a-7dc7-43e1-bcdc-f4f0910bbacc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:51.194788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b777a-7dc7-43e1-bcdc-f4f0910bbacc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:51.196257 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b777a-7dc7-43e1-bcdc-f4f0910bbacc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:51.199720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424e7bc-c3c5-46b0-8f0a-8144f435f2d2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:51.208556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 425/845] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:51.210693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf7e09-66f7-42fb-9912-d9237411d203 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:51.211054 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf7e09-66f7-42fb-9912-d9237411d203 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:51.211839 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf7e09-66f7-42fb-9912-d9237411d203 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:51.213747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 426/846] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:51.214813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf7e09-66f7-42fb-9912-d9237411d203 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:51.239299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f042a3f-c45c-49b5-8346-6c4e45989d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:51.240162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f042a3f-c45c-49b5-8346-6c4e45989d4c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:51.240451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f042a3f-c45c-49b5-8346-6c4e45989d4c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:51.240867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f042a3f-c45c-49b5-8346-6c4e45989d4c None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:51.241710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f042a3f-c45c-49b5-8346-6c4e45989d4c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:51.242023 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f042a3f-c45c-49b5-8346-6c4e45989d4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:51.243590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f042a3f-c45c-49b5-8346-6c4e45989d4c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:51.258401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:51.267979 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 421/847] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Jul 12 03:36:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:51.267979 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98201cfd-b0f0-44b4-ace0-4b19179db89b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:51.267979 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98201cfd-b0f0-44b4-ace0-4b19179db89b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:51.267979 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98201cfd-b0f0-44b4-ace0-4b19179db89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:51.282627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-98201cfd-b0f0-44b4-ace0-4b19179db89b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:51.283904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-98201cfd-b0f0-44b4-ace0-4b19179db89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:51.286289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:51.330397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf7e09-66f7-42fb-9912-d9237411d203 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:51.352668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 422/848] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:36:51.355643 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98201cfd-b0f0-44b4-ace0-4b19179db89b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:51.373221 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0b715-cb53-4315-93cc-ae9c7a83cd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:51.374892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0b715-cb53-4315-93cc-ae9c7a83cd19 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:51.375316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0b715-cb53-4315-93cc-ae9c7a83cd19 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:51.375744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0b715-cb53-4315-93cc-ae9c7a83cd19 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:51.383342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0b715-cb53-4315-93cc-ae9c7a83cd19 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:51.384887 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0b715-cb53-4315-93cc-ae9c7a83cd19 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-b6c0b715-cb53-4315-93cc-ae9c7a83cd19 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:51.399849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:51.406424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:51.407400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:51.407400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:51.407400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:51.407400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:51.407400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:51.407400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:51.407400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:51.407400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:51.407400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 03:36:51.407400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:51.407400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 427/849] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:36:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:51.407400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0b715-cb53-4315-93cc-ae9c7a83cd19 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:51.466048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98201cfd-b0f0-44b4-ace0-4b19179db89b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:51.503020 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a432a0e-d9d1-43ff-8db6-1ee9305b4f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:51.503839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a432a0e-d9d1-43ff-8db6-1ee9305b4f27 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:51.504170 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a432a0e-d9d1-43ff-8db6-1ee9305b4f27 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:51.504757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a432a0e-d9d1-43ff-8db6-1ee9305b4f27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:51.510593 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:51.514056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0b715-cb53-4315-93cc-ae9c7a83cd19 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:51.514278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:51.553538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe698f-cd3f-48ac-bd93-59a31cb212a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:51.554612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe698f-cd3f-48ac-bd93-59a31cb212a1 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:51.555324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe698f-cd3f-48ac-bd93-59a31cb212a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:51.555678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe698f-cd3f-48ac-bd93-59a31cb212a1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:51.556584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe698f-cd3f-48ac-bd93-59a31cb212a1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:51.558072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe698f-cd3f-48ac-bd93-59a31cb212a1 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-f9fe698f-cd3f-48ac-bd93-59a31cb212a1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:51.570121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:51.571248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 03:36:51.572194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:51.572936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 428/850] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:51.573734 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe698f-cd3f-48ac-bd93-59a31cb212a1 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:51.585864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a432a0e-d9d1-43ff-8db6-1ee9305b4f27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:51.590673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a432a0e-d9d1-43ff-8db6-1ee9305b4f27 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:51.693377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe698f-cd3f-48ac-bd93-59a31cb212a1 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:51.698120 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 423/851] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:51 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:36:51.698967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a432a0e-d9d1-43ff-8db6-1ee9305b4f27 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:51.735561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31d530-3b71-40d9-b793-7fe5fef5f3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:51.736252 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31d530-3b71-40d9-b793-7fe5fef5f3e0 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:51.736534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31d530-3b71-40d9-b793-7fe5fef5f3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:51.736803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31d530-3b71-40d9-b793-7fe5fef5f3e0 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:51.737626 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31d530-3b71-40d9-b793-7fe5fef5f3e0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:51.737933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31d530-3b71-40d9-b793-7fe5fef5f3e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:51.739085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:51.739366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31d530-3b71-40d9-b793-7fe5fef5f3e0 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:51.757120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 429/852] 10.222.0.20 () {56 vars in 1135 bytes} [Tue Jul 12 03:36:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:51.757744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31d530-3b71-40d9-b793-7fe5fef5f3e0 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:51.782859 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:51.804358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a432a0e-d9d1-43ff-8db6-1ee9305b4f27 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:51.842908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a06b61-3977-4887-8392-e0d3a5321167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:51.843587 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a06b61-3977-4887-8392-e0d3a5321167 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:51.843872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a06b61-3977-4887-8392-e0d3a5321167 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:51.844178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a06b61-3977-4887-8392-e0d3a5321167 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:51.844891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a06b61-3977-4887-8392-e0d3a5321167 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:51.845200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a06b61-3977-4887-8392-e0d3a5321167 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:51.846600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a06b61-3977-4887-8392-e0d3a5321167 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:51.851312 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 424/853] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:51.851931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a06b61-3977-4887-8392-e0d3a5321167 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:51.864949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31d530-3b71-40d9-b793-7fe5fef5f3e0 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:51.899527 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328d19c-8296-43a8-93d4-b75e08c8d9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:51.900343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328d19c-8296-43a8-93d4-b75e08c8d9bc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:51.900658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328d19c-8296-43a8-93d4-b75e08c8d9bc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:51.900894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328d19c-8296-43a8-93d4-b75e08c8d9bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:51.911798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328d19c-8296-43a8-93d4-b75e08c8d9bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:51.913420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328d19c-8296-43a8-93d4-b75e08c8d9bc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-e328d19c-8296-43a8-93d4-b75e08c8d9bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:51.931443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:51.932413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:51.933317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:51.933317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:51.933317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:51.933317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:51.933317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 430/854] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:51.933596 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328d19c-8296-43a8-93d4-b75e08c8d9bc None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:51.969505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a06b61-3977-4887-8392-e0d3a5321167 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:51.971268 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:52.007399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46068a-d768-4738-b5ac-fcfd3b2c1a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:52.012099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46068a-d768-4738-b5ac-fcfd3b2c1a5f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:52.012422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46068a-d768-4738-b5ac-fcfd3b2c1a5f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:52.012696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46068a-d768-4738-b5ac-fcfd3b2c1a5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:52.014958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:52.039595 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328d19c-8296-43a8-93d4-b75e08c8d9bc None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:52.075288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97d154-103e-4b77-be8b-448734329711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:52.075958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97d154-103e-4b77-be8b-448734329711 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:52.076271 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97d154-103e-4b77-be8b-448734329711 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:52.076548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97d154-103e-4b77-be8b-448734329711 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:52.077328 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97d154-103e-4b77-be8b-448734329711 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:52.077637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97d154-103e-4b77-be8b-448734329711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:52.079019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97d154-103e-4b77-be8b-448734329711 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:52.079991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46068a-d768-4738-b5ac-fcfd3b2c1a5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:52.081455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46068a-d768-4738-b5ac-fcfd3b2c1a5f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:52.083815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 431/855] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:51 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:52.084466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97d154-103e-4b77-be8b-448734329711 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:52.217710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 425/856] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:51 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:36:52.224060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97d154-103e-4b77-be8b-448734329711 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:52.225425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46068a-d768-4738-b5ac-fcfd3b2c1a5f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:52.234141 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:52.261671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d105dc-7849-4644-9070-7f78ab76cc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:52.262517 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d105dc-7849-4644-9070-7f78ab76cc0e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:52.262897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d105dc-7849-4644-9070-7f78ab76cc0e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:52.263273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d105dc-7849-4644-9070-7f78ab76cc0e None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:52.264647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d105dc-7849-4644-9070-7f78ab76cc0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:52.264647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d105dc-7849-4644-9070-7f78ab76cc0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:52.266095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d105dc-7849-4644-9070-7f78ab76cc0e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:52.274209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:52.280136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 432/857] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:52.289193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d105dc-7849-4644-9070-7f78ab76cc0e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:52.345677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46068a-d768-4738-b5ac-fcfd3b2c1a5f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:52.390136 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856fd93-ca28-41a8-ae71-a9dd9b279ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:52.391734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856fd93-ca28-41a8-ae71-a9dd9b279ae6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:52.392292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856fd93-ca28-41a8-ae71-a9dd9b279ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:52.392721 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856fd93-ca28-41a8-ae71-a9dd9b279ae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:52.400288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d105dc-7849-4644-9070-7f78ab76cc0e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:52.406709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856fd93-ca28-41a8-ae71-a9dd9b279ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:52.409017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856fd93-ca28-41a8-ae71-a9dd9b279ae6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-b856fd93-ca28-41a8-ae71-a9dd9b279ae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:52.430504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:52.431629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:52.432730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:52.432730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:52.432730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:52.432730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:52.432730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 426/858] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:52.433289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435cf9f-5f53-46be-9e9f-c8ad95293d0e None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:52.433575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435cf9f-5f53-46be-9e9f-c8ad95293d0e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:52.433850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435cf9f-5f53-46be-9e9f-c8ad95293d0e None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:52.435311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 427/859] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:52.439832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435cf9f-5f53-46be-9e9f-c8ad95293d0e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:52.443480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af9f2c-591b-494d-9234-48ac4a066d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:52.444409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af9f2c-591b-494d-9234-48ac4a066d55 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:52.444757 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af9f2c-591b-494d-9234-48ac4a066d55 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:52.445106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af9f2c-591b-494d-9234-48ac4a066d55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:52.452489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af9f2c-591b-494d-9234-48ac4a066d55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:52.454025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af9f2c-591b-494d-9234-48ac4a066d55 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-86af9f2c-591b-494d-9234-48ac4a066d55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:52.468826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:52.469870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:52.470963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:52.470963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:52.470963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:52.470963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:52.470963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:52.470963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:52.470963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:52.470963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:52.476477 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 433/860] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:36:52.477638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fcb56-4ad8-4c7b-9755-61c00ab1181c None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:52.477994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fcb56-4ad8-4c7b-9755-61c00ab1181c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:52.478357 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fcb56-4ad8-4c7b-9755-61c00ab1181c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:52.495458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-c26fcb56-4ad8-4c7b-9755-61c00ab1181c None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:52.496765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c26fcb56-4ad8-4c7b-9755-61c00ab1181c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:52.501992 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:52.551849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435cf9f-5f53-46be-9e9f-c8ad95293d0e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:52.552899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:52.568044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 434/861] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:52 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:36:52.569163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fcb56-4ad8-4c7b-9755-61c00ab1181c None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:52.596307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096eccc-c5fb-43d3-a496-73050f0ba29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:52.596962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096eccc-c5fb-43d3-a496-73050f0ba29a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:52.597240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096eccc-c5fb-43d3-a496-73050f0ba29a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:52.597515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096eccc-c5fb-43d3-a496-73050f0ba29a None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:52.598254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096eccc-c5fb-43d3-a496-73050f0ba29a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:52.598555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096eccc-c5fb-43d3-a496-73050f0ba29a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:52.599931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096eccc-c5fb-43d3-a496-73050f0ba29a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:52.617755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 428/862] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:52.617755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096eccc-c5fb-43d3-a496-73050f0ba29a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:52.678198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fcb56-4ad8-4c7b-9755-61c00ab1181c None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:52.711812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6defb-cd72-4478-9492-59d016d8336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:52.712511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6defb-cd72-4478-9492-59d016d8336d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:52.712804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6defb-cd72-4478-9492-59d016d8336d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:52.713089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6defb-cd72-4478-9492-59d016d8336d None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:52.713884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6defb-cd72-4478-9492-59d016d8336d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:52.714200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6defb-cd72-4478-9492-59d016d8336d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:52.715659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6defb-cd72-4478-9492-59d016d8336d None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:52.718085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096eccc-c5fb-43d3-a496-73050f0ba29a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:52.732438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 435/863] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:52.732438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6defb-cd72-4478-9492-59d016d8336d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:52.738320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:52.760545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3222d3-fae5-47f8-b378-1c386179c642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:52.761752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3222d3-fae5-47f8-b378-1c386179c642 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:52.762027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3222d3-fae5-47f8-b378-1c386179c642 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:52.762296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3222d3-fae5-47f8-b378-1c386179c642 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:52.762943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3222d3-fae5-47f8-b378-1c386179c642 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:52.764498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3222d3-fae5-47f8-b378-1c386179c642 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-4c3222d3-fae5-47f8-b378-1c386179c642 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:52.777267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:52.778193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:52.779022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 429/864] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Jul 12 03:36:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:52.779889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3222d3-fae5-47f8-b378-1c386179c642 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:52.787082 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:52.840347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6defb-cd72-4478-9492-59d016d8336d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:52.876366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a589f5-05b2-4153-b8ef-c365cfe4ae4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:52.877181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a589f5-05b2-4153-b8ef-c365cfe4ae4a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:52.877445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a589f5-05b2-4153-b8ef-c365cfe4ae4a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:52.877702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a589f5-05b2-4153-b8ef-c365cfe4ae4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:52.884655 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a589f5-05b2-4153-b8ef-c365cfe4ae4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:52.886904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a589f5-05b2-4153-b8ef-c365cfe4ae4a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:52.888701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3222d3-fae5-47f8-b378-1c386179c642 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-f8a589f5-05b2-4153-b8ef-c365cfe4ae4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:52.900931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:52.901820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:52.902682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:52.902682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:52.902682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:52.902682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:52.902682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:52.902682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:52.902682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:52.902682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:52.902682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 436/865] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:52.903153 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a589f5-05b2-4153-b8ef-c365cfe4ae4a None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:52.928621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630ca08-85e1-4f1c-966d-e33457a7240a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:52.929275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630ca08-85e1-4f1c-966d-e33457a7240a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:52.929553 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630ca08-85e1-4f1c-966d-e33457a7240a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:52.929821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630ca08-85e1-4f1c-966d-e33457a7240a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:52.930642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630ca08-85e1-4f1c-966d-e33457a7240a None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=2b01c3b5c3384d4ca0999e9156c3a9cb) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:52.930947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630ca08-85e1-4f1c-966d-e33457a7240a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:52.932416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630ca08-85e1-4f1c-966d-e33457a7240a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:52.944185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 430/866] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:52 2022] GET /identity/v3/role_assignments?user.id=2b01c3b5c3384d4ca0999e9156c3a9cb&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:52.944185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630ca08-85e1-4f1c-966d-e33457a7240a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:53.034174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a589f5-05b2-4153-b8ef-c365cfe4ae4a None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:53.044475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:53.048276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:53.082618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630ca08-85e1-4f1c-966d-e33457a7240a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:53.083879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22ebc7-aa42-4245-80ec-9c8bd24b682c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:53.085087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22ebc7-aa42-4245-80ec-9c8bd24b682c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:53.086503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22ebc7-aa42-4245-80ec-9c8bd24b682c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:53.086981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22ebc7-aa42-4245-80ec-9c8bd24b682c None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:53.087874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22ebc7-aa42-4245-80ec-9c8bd24b682c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:53.088437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22ebc7-aa42-4245-80ec-9c8bd24b682c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:53.090009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22ebc7-aa42-4245-80ec-9c8bd24b682c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:53.098679 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 437/867] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 12 03:36:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:53.104291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22ebc7-aa42-4245-80ec-9c8bd24b682c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:53.142407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116abeb-8568-4b4f-9192-b371963278d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:53.143368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116abeb-8568-4b4f-9192-b371963278d4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:53.143368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116abeb-8568-4b4f-9192-b371963278d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:53.143640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116abeb-8568-4b4f-9192-b371963278d4 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:53.144419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116abeb-8568-4b4f-9192-b371963278d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:53.144727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116abeb-8568-4b4f-9192-b371963278d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:53.146225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116abeb-8568-4b4f-9192-b371963278d4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:53.159932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 431/868] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:53.166129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116abeb-8568-4b4f-9192-b371963278d4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:53.231306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22ebc7-aa42-4245-80ec-9c8bd24b682c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:53.278732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf243fb-fea6-420a-bb65-3969b6c341d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:53.279425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf243fb-fea6-420a-bb65-3969b6c341d1 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:53.279705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf243fb-fea6-420a-bb65-3969b6c341d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:53.280030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf243fb-fea6-420a-bb65-3969b6c341d1 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:53.281034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf243fb-fea6-420a-bb65-3969b6c341d1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:53.281340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf243fb-fea6-420a-bb65-3969b6c341d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:53.283049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf243fb-fea6-420a-bb65-3969b6c341d1 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:53.290288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 438/869] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 12 03:36:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:53.304222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116abeb-8568-4b4f-9192-b371963278d4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:53.306736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:53.345265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc32f2-4d22-4b5e-a370-c920a8de6129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:53.346336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc32f2-4d22-4b5e-a370-c920a8de6129 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:53.346797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc32f2-4d22-4b5e-a370-c920a8de6129 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:53.347316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc32f2-4d22-4b5e-a370-c920a8de6129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:53.348383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc32f2-4d22-4b5e-a370-c920a8de6129 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=0026b8f11158492b90da3395cc2c43c6) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:53.348887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc32f2-4d22-4b5e-a370-c920a8de6129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:53.350550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc32f2-4d22-4b5e-a370-c920a8de6129 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:53.358142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 432/870] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:53 2022] GET /identity/v3/role_assignments?user.id=0026b8f11158492b90da3395cc2c43c6&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:54.164597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27edef-9f09-4fd1-8bb0-05f1b592c1c3 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:54.165675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27edef-9f09-4fd1-8bb0-05f1b592c1c3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:54.168098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27edef-9f09-4fd1-8bb0-05f1b592c1c3 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:54.170263 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 439/871] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:54.176673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c07ed-95e9-4e6f-9eaf-58916fb54cfd None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:54.177344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c07ed-95e9-4e6f-9eaf-58916fb54cfd None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:54.177826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c07ed-95e9-4e6f-9eaf-58916fb54cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:54.195831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e19c07ed-95e9-4e6f-9eaf-58916fb54cfd None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:54.197425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e19c07ed-95e9-4e6f-9eaf-58916fb54cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:54.266294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 433/872] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:36:54.288942 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27edef-9f09-4fd1-8bb0-05f1b592c1c3 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:54.327104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:54.394415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27edef-9f09-4fd1-8bb0-05f1b592c1c3 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:54.430878 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbade3-3600-4100-b756-550edc9f7f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:54.431545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbade3-3600-4100-b756-550edc9f7f9f None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:54.431825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbade3-3600-4100-b756-550edc9f7f9f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:54.432123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbade3-3600-4100-b756-550edc9f7f9f None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:54.432849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbade3-3600-4100-b756-550edc9f7f9f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:54.434219 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbade3-3600-4100-b756-550edc9f7f9f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:54.449700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 440/873] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:36:54 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:36:55.121874 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781978-dacd-4db8-be43-49f125b8cb62 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:55.122433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781978-dacd-4db8-be43-49f125b8cb62 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:55.122433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781978-dacd-4db8-be43-49f125b8cb62 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:55.124071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 434/874] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:55.129327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcab2a-45e3-4df9-84e0-26dcee0301f3 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:55.129622 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcab2a-45e3-4df9-84e0-26dcee0301f3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:55.129899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcab2a-45e3-4df9-84e0-26dcee0301f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:55.147556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-10fcab2a-45e3-4df9-84e0-26dcee0301f3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:55.148766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-10fcab2a-45e3-4df9-84e0-26dcee0301f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:55.225743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 441/875] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:55 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:36:55.248183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05781978-dacd-4db8-be43-49f125b8cb62 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:55.347909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0ae58-4ed4-49cc-8fe1-d0e7f8b56270 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:55.348391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0ae58-4ed4-49cc-8fe1-d0e7f8b56270 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:55.348764 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0ae58-4ed4-49cc-8fe1-d0e7f8b56270 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:55.350290 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:55.350519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 442/876] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:55.355831 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d7744-9e60-478d-9179-0040fb990837 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:55.356362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d7744-9e60-478d-9179-0040fb990837 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:55.357015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d7744-9e60-478d-9179-0040fb990837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:55.358733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05781978-dacd-4db8-be43-49f125b8cb62 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:55.373964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-e19d7744-9e60-478d-9179-0040fb990837 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:55.375252 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e19d7744-9e60-478d-9179-0040fb990837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:55.392577 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:55.397287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4170e2-8bad-4519-a695-56c19637e64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:55.398327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4170e2-8bad-4519-a695-56c19637e64b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:55.399251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4170e2-8bad-4519-a695-56c19637e64b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:55.399734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4170e2-8bad-4519-a695-56c19637e64b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:55.412918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4170e2-8bad-4519-a695-56c19637e64b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-9f4170e2-8bad-4519-a695-56c19637e64b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:36:55.421941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:55.422984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:55.424029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:55.424029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:55.424029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:55.424029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:55.424029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:36:55.424029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:36:55.424029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:36:55.424029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:36:55.424029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:36:55.424029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:36:55.424029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 03:36:55.424029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:55.426510 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 435/877] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Jul 12 03:36:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:55.436061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4170e2-8bad-4519-a695-56c19637e64b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:55.447814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 443/878] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:55 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:36:55.457419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89520d39-3409-4e2a-8953-0e07dc94a765 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:55.457708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89520d39-3409-4e2a-8953-0e07dc94a765 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:55.457982 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89520d39-3409-4e2a-8953-0e07dc94a765 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:55.459418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 444/879] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:55 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:55.465087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c1431-851e-4a59-80c5-a4e90d83e4a9 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:55.465573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c1431-851e-4a59-80c5-a4e90d83e4a9 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:55.466405 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c1431-851e-4a59-80c5-a4e90d83e4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:55.483861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-906c1431-851e-4a59-80c5-a4e90d83e4a9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:55.484987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-906c1431-851e-4a59-80c5-a4e90d83e4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:55.549528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4170e2-8bad-4519-a695-56c19637e64b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:55.553214 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 445/880] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:55 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:36:55.554460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c1431-851e-4a59-80c5-a4e90d83e4a9 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:55.588020 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302511d-d45a-4f86-8fba-1642748c5df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:55.588749 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302511d-d45a-4f86-8fba-1642748c5df4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:55.589029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302511d-d45a-4f86-8fba-1642748c5df4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:55.589301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302511d-d45a-4f86-8fba-1642748c5df4 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:55.590046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302511d-d45a-4f86-8fba-1642748c5df4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:55.590349 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302511d-d45a-4f86-8fba-1642748c5df4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:55.591737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302511d-d45a-4f86-8fba-1642748c5df4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:55.600806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 436/881] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 12 03:36:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:55.601526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302511d-d45a-4f86-8fba-1642748c5df4 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:55.620780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:55.658039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:55.679387 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c1431-851e-4a59-80c5-a4e90d83e4a9 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:55.711690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302511d-d45a-4f86-8fba-1642748c5df4 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:55.717158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28adc0-2e03-4d28-9bc3-ddacde02fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:55.717825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28adc0-2e03-4d28-9bc3-ddacde02fa2e None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:55.718108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28adc0-2e03-4d28-9bc3-ddacde02fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:55.718383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28adc0-2e03-4d28-9bc3-ddacde02fa2e None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:55.719575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28adc0-2e03-4d28-9bc3-ddacde02fa2e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:55.721424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28adc0-2e03-4d28-9bc3-ddacde02fa2e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:55.737702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 446/882] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:36:55 2022] POST /identity/v3/services => generated 269 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:36:55.738431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28adc0-2e03-4d28-9bc3-ddacde02fa2e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:55.765239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1c50c-d88b-4bb8-9e24-d133a3102f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:55.766787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1c50c-d88b-4bb8-9e24-d133a3102f32 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:55.767654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1c50c-d88b-4bb8-9e24-d133a3102f32 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:55.768161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1c50c-d88b-4bb8-9e24-d133a3102f32 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:55.770070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1c50c-d88b-4bb8-9e24-d133a3102f32 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:55.772605 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe1c50c-d88b-4bb8-9e24-d133a3102f32 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-dbe1c50c-d88b-4bb8-9e24-d133a3102f32 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:55.792967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:55.794011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:36:55.794852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:55.802914 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 437/883] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 12 03:36:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:55.805365 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1c50c-d88b-4bb8-9e24-d133a3102f32 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:55.862083 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28adc0-2e03-4d28-9bc3-ddacde02fa2e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:55.899171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88b1f7-1890-4b6e-9904-06db07141fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:55.900096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88b1f7-1890-4b6e-9904-06db07141fac None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:55.900360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88b1f7-1890-4b6e-9904-06db07141fac None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:55.900630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88b1f7-1890-4b6e-9904-06db07141fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:55.901071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:55.913746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88b1f7-1890-4b6e-9904-06db07141fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:55.915198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88b1f7-1890-4b6e-9904-06db07141fac None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:55.917157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:55.928934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1c50c-d88b-4bb8-9e24-d133a3102f32 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-7d88b1f7-1890-4b6e-9904-06db07141fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:55.935321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:55.936324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:55.937395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:55.937395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:55.937395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:55.937395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:55.937395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 447/884] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:55.943341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88b1f7-1890-4b6e-9904-06db07141fac None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:55.980055 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24be669-f957-4c15-bf48-796ac1b58738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:55.980771 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24be669-f957-4c15-bf48-796ac1b58738 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:55.981335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24be669-f957-4c15-bf48-796ac1b58738 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:55.981609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24be669-f957-4c15-bf48-796ac1b58738 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:55.982701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24be669-f957-4c15-bf48-796ac1b58738 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:55.983051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24be669-f957-4c15-bf48-796ac1b58738 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:55.984467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24be669-f957-4c15-bf48-796ac1b58738 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:55.990888 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 438/885] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 12 03:36:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:55.996416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24be669-f957-4c15-bf48-796ac1b58738 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:56.055653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88b1f7-1890-4b6e-9904-06db07141fac None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:56.092456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bd0fe-77a8-4c35-858a-b6fbab61885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:56.093158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bd0fe-77a8-4c35-858a-b6fbab61885b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:56.093442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bd0fe-77a8-4c35-858a-b6fbab61885b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:56.093720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bd0fe-77a8-4c35-858a-b6fbab61885b None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:56.094443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34bd0fe-77a8-4c35-858a-b6fbab61885b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:56.094744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34bd0fe-77a8-4c35-858a-b6fbab61885b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:56.097919 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34bd0fe-77a8-4c35-858a-b6fbab61885b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:56.114494 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 448/886] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:56.119702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bd0fe-77a8-4c35-858a-b6fbab61885b None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:56.128737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24be669-f957-4c15-bf48-796ac1b58738 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:56.166821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47df77-bddd-4985-a169-9e77292b613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:56.167544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47df77-bddd-4985-a169-9e77292b613e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:56.167828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47df77-bddd-4985-a169-9e77292b613e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:56.168161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47df77-bddd-4985-a169-9e77292b613e None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:56.168911 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47df77-bddd-4985-a169-9e77292b613e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:56.169217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47df77-bddd-4985-a169-9e77292b613e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:56.170625 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd47df77-bddd-4985-a169-9e77292b613e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:56.177387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 439/887] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 12 03:36:55 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:56.195005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:56.245641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bd0fe-77a8-4c35-858a-b6fbab61885b None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:56.284684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593b833-b888-4154-b567-ced098dfd487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:56.285871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593b833-b888-4154-b567-ced098dfd487 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:56.286350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593b833-b888-4154-b567-ced098dfd487 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:56.286822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593b833-b888-4154-b567-ced098dfd487 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:56.295506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593b833-b888-4154-b567-ced098dfd487 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:56.297438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593b833-b888-4154-b567-ced098dfd487 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-d593b833-b888-4154-b567-ced098dfd487 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:36:56.314618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:56.315631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:56.316604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:56.316604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:56.316604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:36:56.316604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:36:56.316604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:36:56.316604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:36:56.316604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:36:56.316604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:36:56.316604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 03:36:56.316604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:56.319369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 449/888] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 12 03:36:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:56.325402 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47df77-bddd-4985-a169-9e77292b613e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:56.335856 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:56.437769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47df77-bddd-4985-a169-9e77292b613e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:56.473075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f45c27-a828-4e24-99e5-3ba133b42837 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:56.473388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f45c27-a828-4e24-99e5-3ba133b42837 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:56.473638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f45c27-a828-4e24-99e5-3ba133b42837 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:56.475258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 450/889] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:56.475818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08aaf3e-0a21-4f5a-a69f-c2eca814a6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:56.476743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08aaf3e-0a21-4f5a-a69f-c2eca814a6d1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:56.477216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08aaf3e-0a21-4f5a-a69f-c2eca814a6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:56.477677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08aaf3e-0a21-4f5a-a69f-c2eca814a6d1 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:56.478769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08aaf3e-0a21-4f5a-a69f-c2eca814a6d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:56.479317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08aaf3e-0a21-4f5a-a69f-c2eca814a6d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:56.480169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49501e1-ffc9-48cb-90ad-ce3eae2dc1ae None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:56.480461 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49501e1-ffc9-48cb-90ad-ce3eae2dc1ae None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:56.480739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49501e1-ffc9-48cb-90ad-ce3eae2dc1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:56.481574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08aaf3e-0a21-4f5a-a69f-c2eca814a6d1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:56.498096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-b49501e1-ffc9-48cb-90ad-ce3eae2dc1ae None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:56.499228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b49501e1-ffc9-48cb-90ad-ce3eae2dc1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:56.504057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 440/890] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Jul 12 03:36:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:56.509716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08aaf3e-0a21-4f5a-a69f-c2eca814a6d1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:56.584093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 451/891] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:36:56.591717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:56.603391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49501e1-ffc9-48cb-90ad-ce3eae2dc1ae None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:56.633676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08aaf3e-0a21-4f5a-a69f-c2eca814a6d1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:56.662750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:56.671554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f7dc3-9cda-4226-b60d-3cc8612d0cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:56.672655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f7dc3-9cda-4226-b60d-3cc8612d0cca None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:56.673139 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f7dc3-9cda-4226-b60d-3cc8612d0cca None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:56.673607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f7dc3-9cda-4226-b60d-3cc8612d0cca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:56.726645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49501e1-ffc9-48cb-90ad-ce3eae2dc1ae None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:56.748468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f7dc3-9cda-4226-b60d-3cc8612d0cca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:56.750308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f7dc3-9cda-4226-b60d-3cc8612d0cca None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:56.763426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405660e5-642c-41f7-a811-b95146b28cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:56.764301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405660e5-642c-41f7-a811-b95146b28cea None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:56.764584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405660e5-642c-41f7-a811-b95146b28cea None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:56.764857 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405660e5-642c-41f7-a811-b95146b28cea None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:56.765507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405660e5-642c-41f7-a811-b95146b28cea None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:56.766964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405660e5-642c-41f7-a811-b95146b28cea None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-405660e5-642c-41f7-a811-b95146b28cea None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:56.779248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:56.780322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:56.786481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 452/892] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:56.787282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405660e5-642c-41f7-a811-b95146b28cea None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:56.828751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:56.871758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 441/893] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:36:56 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 03:36:56.877791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f7dc3-9cda-4226-b60d-3cc8612d0cca None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:56.897416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405660e5-642c-41f7-a811-b95146b28cea None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:56.907087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:56.935324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e6668-b5a6-4398-be51-d8ed1071d87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:56.936106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e6668-b5a6-4398-be51-d8ed1071d87a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:56.936398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e6668-b5a6-4398-be51-d8ed1071d87a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:56.936677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e6668-b5a6-4398-be51-d8ed1071d87a None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:56.937626 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e6668-b5a6-4398-be51-d8ed1071d87a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:56.937735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e6668-b5a6-4398-be51-d8ed1071d87a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:56.939619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e6668-b5a6-4398-be51-d8ed1071d87a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:56.951823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 453/894] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:56.953342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e6668-b5a6-4398-be51-d8ed1071d87a None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:57.022363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f7dc3-9cda-4226-b60d-3cc8612d0cca None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:57.064182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586858b-4fa5-4ab0-8a6b-dd1a5ca24e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:57.065672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586858b-4fa5-4ab0-8a6b-dd1a5ca24e16 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:57.066167 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586858b-4fa5-4ab0-8a6b-dd1a5ca24e16 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:57.066670 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586858b-4fa5-4ab0-8a6b-dd1a5ca24e16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:57.069828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e6668-b5a6-4398-be51-d8ed1071d87a None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:57.080490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586858b-4fa5-4ab0-8a6b-dd1a5ca24e16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:57.082257 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586858b-4fa5-4ab0-8a6b-dd1a5ca24e16 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-4586858b-4fa5-4ab0-8a6b-dd1a5ca24e16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:36:57.100729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:57.101645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:57.102530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:36:57.102530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:36:57.102530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:36:57.102530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:57.108988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd49b6-1980-4d34-834d-74c7ac0d10cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:57.109155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd49b6-1980-4d34-834d-74c7ac0d10cd None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:57.109373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd49b6-1980-4d34-834d-74c7ac0d10cd None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:57.109573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 442/895] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:36:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:57.109671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd49b6-1980-4d34-834d-74c7ac0d10cd None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:57.110304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd49b6-1980-4d34-834d-74c7ac0d10cd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:57.110609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd49b6-1980-4d34-834d-74c7ac0d10cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:57.111209 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586858b-4fa5-4ab0-8a6b-dd1a5ca24e16 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:57.111976 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd49b6-1980-4d34-834d-74c7ac0d10cd None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:57.124092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 454/896] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:57.127996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd49b6-1980-4d34-834d-74c7ac0d10cd None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:57.134890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:57.169063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:57.237412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd49b6-1980-4d34-834d-74c7ac0d10cd None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:57.246152 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586858b-4fa5-4ab0-8a6b-dd1a5ca24e16 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:57.273456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4b412-0441-4054-9ac3-e52da5924806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:57.275019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4b412-0441-4054-9ac3-e52da5924806 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:57.275562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4b412-0441-4054-9ac3-e52da5924806 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:57.276171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4b412-0441-4054-9ac3-e52da5924806 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:57.277276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4b412-0441-4054-9ac3-e52da5924806 None admin] RBAC: Adding query filter params (interface=public, service_id=c4ae77280deb4c37ae51bb2654193e19, region_id=RegionOne) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:57.277837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4b412-0441-4054-9ac3-e52da5924806 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:57.279540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a4b412-0441-4054-9ac3-e52da5924806 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:57.285610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 455/897] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 12 03:36:57 2022] GET /identity/v3/endpoints?service_id=c4ae77280deb4c37ae51bb2654193e19&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:57.290729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4b412-0441-4054-9ac3-e52da5924806 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:57.291654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944be154-91fd-442f-9b63-f7eee297fc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:57.292635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944be154-91fd-442f-9b63-f7eee297fc4a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:57.293121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944be154-91fd-442f-9b63-f7eee297fc4a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:57.293609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944be154-91fd-442f-9b63-f7eee297fc4a None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:57.294545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944be154-91fd-442f-9b63-f7eee297fc4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:57.295105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944be154-91fd-442f-9b63-f7eee297fc4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:57.296715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944be154-91fd-442f-9b63-f7eee297fc4a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:57.312055 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 443/898] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:57.314822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944be154-91fd-442f-9b63-f7eee297fc4a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:57.409970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:57.413413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4b412-0441-4054-9ac3-e52da5924806 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:57.417952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:57.448821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944be154-91fd-442f-9b63-f7eee297fc4a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:57.462866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c7203-124a-4115-9fcf-b39f1b919122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:57.463814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c7203-124a-4115-9fcf-b39f1b919122 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:57.464130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c7203-124a-4115-9fcf-b39f1b919122 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:57.464408 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c7203-124a-4115-9fcf-b39f1b919122 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:57.465592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c7203-124a-4115-9fcf-b39f1b919122 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:57.467369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c7203-124a-4115-9fcf-b39f1b919122 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:57.476036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 456/899] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 12 03:36:57 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:36:57.493127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f896a-13ea-4e92-9897-33217d2170c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:57.494325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f896a-13ea-4e92-9897-33217d2170c0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:57.494805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f896a-13ea-4e92-9897-33217d2170c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:57.495263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f896a-13ea-4e92-9897-33217d2170c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:57.504122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f896a-13ea-4e92-9897-33217d2170c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:57.505959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f896a-13ea-4e92-9897-33217d2170c0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-fb1f896a-13ea-4e92-9897-33217d2170c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:36:57.522287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:57.523226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:57.524204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:36:57.524204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:36:57.524204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:36:57.524204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:36:57.524204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:36:57.524204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:36:57.524204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:36:57.524204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:57.526627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 444/900] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:36:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:57.533264 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c7203-124a-4115-9fcf-b39f1b919122 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:57.645367 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c7203-124a-4115-9fcf-b39f1b919122 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:57.680033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae698d6-2a65-4ab3-be88-a09173494c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:57.680761 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae698d6-2a65-4ab3-be88-a09173494c1a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:57.681049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae698d6-2a65-4ab3-be88-a09173494c1a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:57.681325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae698d6-2a65-4ab3-be88-a09173494c1a None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:57.682114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae698d6-2a65-4ab3-be88-a09173494c1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:57.682425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae698d6-2a65-4ab3-be88-a09173494c1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:57.683873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae698d6-2a65-4ab3-be88-a09173494c1a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:57.696690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 457/901] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:36:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:57.705136 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f896a-13ea-4e92-9897-33217d2170c0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:57.814913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f896a-13ea-4e92-9897-33217d2170c0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:57.850864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36409d-c079-4bef-8022-d3c84c01385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:57.851528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36409d-c079-4bef-8022-d3c84c01385e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:57.851812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36409d-c079-4bef-8022-d3c84c01385e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:57.852117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36409d-c079-4bef-8022-d3c84c01385e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:57.852954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36409d-c079-4bef-8022-d3c84c01385e None admin] RBAC: Adding query filter params (role.id=eea62fc4e0174ba39b800050ebe7ff5d, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=2b01c3b5c3384d4ca0999e9156c3a9cb) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:57.853265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36409d-c079-4bef-8022-d3c84c01385e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:57.854715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36409d-c079-4bef-8022-d3c84c01385e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:57.857769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:57.862948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 445/902] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:36:57 2022] GET /identity/v3/role_assignments?user.id=2b01c3b5c3384d4ca0999e9156c3a9cb&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=eea62fc4e0174ba39b800050ebe7ff5d => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:57.903075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b3b57-9a8a-49c4-aeb6-85ef58482b6b None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:57.903391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b3b57-9a8a-49c4-aeb6-85ef58482b6b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:57.903641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b3b57-9a8a-49c4-aeb6-85ef58482b6b None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:57.905453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 458/903] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:57.911158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd12ad2-4906-4754-8d0d-b8e2574cdfe5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:57.911449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd12ad2-4906-4754-8d0d-b8e2574cdfe5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:57.911731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd12ad2-4906-4754-8d0d-b8e2574cdfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:57.930354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4fd12ad2-4906-4754-8d0d-b8e2574cdfe5 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:57.931571 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4fd12ad2-4906-4754-8d0d-b8e2574cdfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:58.009671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 446/904] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:36:58.032544 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b3b57-9a8a-49c4-aeb6-85ef58482b6b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:58.064380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:58.139795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b3b57-9a8a-49c4-aeb6-85ef58482b6b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:58.174203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73778155-32f2-4c42-8480-35ab680a33c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:58.175018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73778155-32f2-4c42-8480-35ab680a33c6 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:58.175303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73778155-32f2-4c42-8480-35ab680a33c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:58.175577 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73778155-32f2-4c42-8480-35ab680a33c6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:58.176252 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73778155-32f2-4c42-8480-35ab680a33c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:58.177637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73778155-32f2-4c42-8480-35ab680a33c6 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-73778155-32f2-4c42-8480-35ab680a33c6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:58.190160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:58.191120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:36:58.191941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 459/905] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Jul 12 03:36:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:58.197569 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd12ad2-4906-4754-8d0d-b8e2574cdfe5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:58.298083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd12ad2-4906-4754-8d0d-b8e2574cdfe5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:58.326981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:58.333395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bcdb8-ec80-4a33-9d25-060ad1e3e8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:58.334116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bcdb8-ec80-4a33-9d25-060ad1e3e8f4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:58.334397 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bcdb8-ec80-4a33-9d25-060ad1e3e8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:58.334720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bcdb8-ec80-4a33-9d25-060ad1e3e8f4 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:58.335433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bcdb8-ec80-4a33-9d25-060ad1e3e8f4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:58.335737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bcdb8-ec80-4a33-9d25-060ad1e3e8f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:58.337160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bcdb8-ec80-4a33-9d25-060ad1e3e8f4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:58.346747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 447/906] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 12 03:36:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:58.348103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73778155-32f2-4c42-8480-35ab680a33c6 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:58.387968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e8d86-2f19-48a8-a1e6-2055421268b5 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:58.388305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e8d86-2f19-48a8-a1e6-2055421268b5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:58.388554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e8d86-2f19-48a8-a1e6-2055421268b5 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:58.390268 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 448/907] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:58.395221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8abc55-98db-4fa9-abb1-bf0e6e1d0208 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:58.395512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8abc55-98db-4fa9-abb1-bf0e6e1d0208 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:58.395788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8abc55-98db-4fa9-abb1-bf0e6e1d0208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:58.412322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dc8abc55-98db-4fa9-abb1-bf0e6e1d0208 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:58.413521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dc8abc55-98db-4fa9-abb1-bf0e6e1d0208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:58.447941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:58.462011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73778155-32f2-4c42-8480-35ab680a33c6 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:58.496147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 449/908] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:36:58.507339 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e891d8e-e70f-456c-9e0e-7bc5005272d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:58.508034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e891d8e-e70f-456c-9e0e-7bc5005272d8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:58.508413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e891d8e-e70f-456c-9e0e-7bc5005272d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:58.508757 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e891d8e-e70f-456c-9e0e-7bc5005272d8 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:58.509472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e891d8e-e70f-456c-9e0e-7bc5005272d8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:58.509828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e891d8e-e70f-456c-9e0e-7bc5005272d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:58.511587 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e891d8e-e70f-456c-9e0e-7bc5005272d8 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:58.517148 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 460/909] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 12 03:36:58 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:36:58.519567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8abc55-98db-4fa9-abb1-bf0e6e1d0208 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:58.522538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e891d8e-e70f-456c-9e0e-7bc5005272d8 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:58.623453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e891d8e-e70f-456c-9e0e-7bc5005272d8 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:58.623838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8abc55-98db-4fa9-abb1-bf0e6e1d0208 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:58.650498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:58.662985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9cd43-66ff-49b5-bb5c-4ccbc3f11ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:58.664073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9cd43-66ff-49b5-bb5c-4ccbc3f11ed9 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:58.664565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9cd43-66ff-49b5-bb5c-4ccbc3f11ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:58.665063 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e32a14-1c32-4959-883b-298e5bbf5888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:58.665386 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9cd43-66ff-49b5-bb5c-4ccbc3f11ed9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:58.665738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e32a14-1c32-4959-883b-298e5bbf5888 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:58.666020 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e32a14-1c32-4959-883b-298e5bbf5888 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:58.666293 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e32a14-1c32-4959-883b-298e5bbf5888 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:58.667018 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e32a14-1c32-4959-883b-298e5bbf5888 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:58.667124 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9cd43-66ff-49b5-bb5c-4ccbc3f11ed9 None admin] RBAC: Adding query filter params (interface=public, service_id=b00a7f915db04e38a08a5648dd9ecb75, region_id=RegionOne) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:36:58.667622 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9cd43-66ff-49b5-bb5c-4ccbc3f11ed9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:58.668484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e32a14-1c32-4959-883b-298e5bbf5888 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:58.669425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9cd43-66ff-49b5-bb5c-4ccbc3f11ed9 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:58.675969 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 461/910] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 12 03:36:58 2022] GET /identity/v3/endpoints?service_id=b00a7f915db04e38a08a5648dd9ecb75&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:36:58.682701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9cd43-66ff-49b5-bb5c-4ccbc3f11ed9 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:58.696040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 450/911] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:36:58 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:36:58.707602 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:58.805077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9cd43-66ff-49b5-bb5c-4ccbc3f11ed9 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:58.849261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746778c-fd08-43bd-b172-edf79d859538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:58.849923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746778c-fd08-43bd-b172-edf79d859538 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:58.850206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746778c-fd08-43bd-b172-edf79d859538 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:58.850529 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746778c-fd08-43bd-b172-edf79d859538 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:58.851206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746778c-fd08-43bd-b172-edf79d859538 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:58.852695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746778c-fd08-43bd-b172-edf79d859538 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:58.859979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 462/912] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 12 03:36:58 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:36:59.584171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126d8ed-60d1-439a-8eaa-6a6a5294dba4 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:59.584802 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126d8ed-60d1-439a-8eaa-6a6a5294dba4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:59.584802 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126d8ed-60d1-439a-8eaa-6a6a5294dba4 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:59.586312 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 451/913] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:36:59.591687 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73e577-e3d5-4009-8a94-6c8a031d3913 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:59.592122 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73e577-e3d5-4009-8a94-6c8a031d3913 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:59.592291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73e577-e3d5-4009-8a94-6c8a031d3913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:59.609386 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2a73e577-e3d5-4009-8a94-6c8a031d3913 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:59.610586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2a73e577-e3d5-4009-8a94-6c8a031d3913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:36:59.681772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 463/914] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:36:59.703995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126d8ed-60d1-439a-8eaa-6a6a5294dba4 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:59.811488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126d8ed-60d1-439a-8eaa-6a6a5294dba4 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:36:59.846891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6886f-bc62-4c1c-b5ca-32ae883f8e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:36:59.848100 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6886f-bc62-4c1c-b5ca-32ae883f8e34 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:59.848628 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6886f-bc62-4c1c-b5ca-32ae883f8e34 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:59.849126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6886f-bc62-4c1c-b5ca-32ae883f8e34 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:59.850056 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6886f-bc62-4c1c-b5ca-32ae883f8e34 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:36:59.851680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6886f-bc62-4c1c-b5ca-32ae883f8e34 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e3d6886f-bc62-4c1c-b5ca-32ae883f8e34 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:36:59.863766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:36:59.864883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 03:36:59.865800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:36:59.869020 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 452/915] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:36:59.877227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73e577-e3d5-4009-8a94-6c8a031d3913 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:36:59.887166 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:36:59.941411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a039656-fd59-44c3-adb1-a827f7b73b71 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:59.941411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a039656-fd59-44c3-adb1-a827f7b73b71 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:59.941411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a039656-fd59-44c3-adb1-a827f7b73b71 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:59.943551 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 453/916] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:36:59.947698 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c7a17-84c1-4784-a661-41536d43be46 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:36:59.947986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c7a17-84c1-4784-a661-41536d43be46 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:36:59.948289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c7a17-84c1-4784-a661-41536d43be46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:36:59.966530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0c4c7a17-84c1-4784-a661-41536d43be46 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:36:59.967975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0c4c7a17-84c1-4784-a661-41536d43be46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:00.005820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73e577-e3d5-4009-8a94-6c8a031d3913 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:00.025536 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:00.045247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 454/917] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:36:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:37:00.048727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163e5c9-5453-4e56-a061-a21311527f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:00.049406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163e5c9-5453-4e56-a061-a21311527f13 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:00.049691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163e5c9-5453-4e56-a061-a21311527f13 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:00.049970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163e5c9-5453-4e56-a061-a21311527f13 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:00.050680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163e5c9-5453-4e56-a061-a21311527f13 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:00.050987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163e5c9-5453-4e56-a061-a21311527f13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:00.052487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163e5c9-5453-4e56-a061-a21311527f13 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:00.057814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 464/918] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:36:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:00.063235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c7a17-84c1-4784-a661-41536d43be46 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:00.066656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163e5c9-5453-4e56-a061-a21311527f13 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:00.147321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:00.178282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163e5c9-5453-4e56-a061-a21311527f13 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:00.194821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c7a17-84c1-4784-a661-41536d43be46 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:00.213585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcc3e6-c4b6-4925-aefb-6954089319a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:00.214537 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcc3e6-c4b6-4925-aefb-6954089319a1 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:00.214891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcc3e6-c4b6-4925-aefb-6954089319a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:00.215242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcc3e6-c4b6-4925-aefb-6954089319a1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:00.225827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcc3e6-c4b6-4925-aefb-6954089319a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:37:00.232179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe58127-f64b-4aeb-89ac-b8dec28a0518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:00.232894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe58127-f64b-4aeb-89ac-b8dec28a0518 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:00.233180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe58127-f64b-4aeb-89ac-b8dec28a0518 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:00.233460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe58127-f64b-4aeb-89ac-b8dec28a0518 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:00.234164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe58127-f64b-4aeb-89ac-b8dec28a0518 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-85dcc3e6-c4b6-4925-aefb-6954089319a1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:37:00.234400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:00.235314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:00.236388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:00.236388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:00.236388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:00.236388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:00.236388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:37:00.236388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:37:00.236388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:37:00.236388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:37:00.236388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:37:00.236388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:37:00.236388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 03:37:00.236388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:00.237183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe58127-f64b-4aeb-89ac-b8dec28a0518 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:00.237183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe58127-f64b-4aeb-89ac-b8dec28a0518 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:00.237351 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 465/919] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Jul 12 03:37:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:00.241253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 455/920] 10.222.0.20 () {56 vars in 1114 bytes} [Tue Jul 12 03:37:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:00.244958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcc3e6-c4b6-4925-aefb-6954089319a1 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:00.249130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe58127-f64b-4aeb-89ac-b8dec28a0518 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:00.309870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:00.355781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcc3e6-c4b6-4925-aefb-6954089319a1 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:00.377888 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe58127-f64b-4aeb-89ac-b8dec28a0518 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:00.388857 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:00.396384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b08cc-1cf3-4d55-a80c-a8b3ae8004f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:00.397154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b08cc-1cf3-4d55-a80c-a8b3ae8004f3 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:00.397496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b08cc-1cf3-4d55-a80c-a8b3ae8004f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:00.397735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b08cc-1cf3-4d55-a80c-a8b3ae8004f3 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:00.399058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b08cc-1cf3-4d55-a80c-a8b3ae8004f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:00.399554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b08cc-1cf3-4d55-a80c-a8b3ae8004f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:00.401322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b08cc-1cf3-4d55-a80c-a8b3ae8004f3 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:00.411074 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 466/921] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 12 03:37:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:00.416612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b08cc-1cf3-4d55-a80c-a8b3ae8004f3 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:00.417853 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee73da-a42e-4fca-8d02-36595e0278d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:00.418527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee73da-a42e-4fca-8d02-36595e0278d0 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:00.418811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee73da-a42e-4fca-8d02-36595e0278d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:00.419089 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee73da-a42e-4fca-8d02-36595e0278d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:00.419880 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee73da-a42e-4fca-8d02-36595e0278d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:00.421296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee73da-a42e-4fca-8d02-36595e0278d0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:00.457094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 456/922] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 12 03:37:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:37:00.541942 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b08cc-1cf3-4d55-a80c-a8b3ae8004f3 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:00.587255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6528dd-5205-4879-90ba-2fc7b2fea00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:00.588423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6528dd-5205-4879-90ba-2fc7b2fea00f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:00.588864 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6528dd-5205-4879-90ba-2fc7b2fea00f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:00.589286 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6528dd-5205-4879-90ba-2fc7b2fea00f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:00.602150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6528dd-5205-4879-90ba-2fc7b2fea00f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:00.604357 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6528dd-5205-4879-90ba-2fc7b2fea00f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-fc6528dd-5205-4879-90ba-2fc7b2fea00f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:37:00.622729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:00.624067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:00.624972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:37:00.624972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:37:00.624972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:00.624972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:00.626520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 467/923] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:37:00.631581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee73da-a42e-4fca-8d02-36595e0278d0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:00.733316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:00.737515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee73da-a42e-4fca-8d02-36595e0278d0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:00.772555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9069089-7209-4243-8597-7a6f128dabff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:00.773232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9069089-7209-4243-8597-7a6f128dabff None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:00.773519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9069089-7209-4243-8597-7a6f128dabff None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:00.773798 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9069089-7209-4243-8597-7a6f128dabff None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:00.774528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9069089-7209-4243-8597-7a6f128dabff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:00.774836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9069089-7209-4243-8597-7a6f128dabff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:00.776366 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9069089-7209-4243-8597-7a6f128dabff None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:00.788957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 457/924] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:00.794623 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6528dd-5205-4879-90ba-2fc7b2fea00f None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:00.838747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:00.899006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6528dd-5205-4879-90ba-2fc7b2fea00f None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:00.912164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ada69-e83c-4a5e-a9c8-cf672deb194f None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:00.912454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ada69-e83c-4a5e-a9c8-cf672deb194f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:00.912920 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ada69-e83c-4a5e-a9c8-cf672deb194f None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:00.914274 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 458/925] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:00.915528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ca123-2d1a-47fc-ae33-8a8848c3975d None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:00.916102 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ca123-2d1a-47fc-ae33-8a8848c3975d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:00.916588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ca123-2d1a-47fc-ae33-8a8848c3975d None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:00.918182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 459/926] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:00.922116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fe2dd-bef3-49e6-aae8-c5a7ca681f2c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:00.922116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fe2dd-bef3-49e6-aae8-c5a7ca681f2c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:00.922116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fe2dd-bef3-49e6-aae8-c5a7ca681f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:00.936884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe52b0-8b11-4bd4-8084-0cf3d8524567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:00.937996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe52b0-8b11-4bd4-8084-0cf3d8524567 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:00.938469 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe52b0-8b11-4bd4-8084-0cf3d8524567 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:00.938581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-067fe2dd-bef3-49e6-aae8-c5a7ca681f2c None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:00.939012 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe52b0-8b11-4bd4-8084-0cf3d8524567 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:00.939667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-067fe2dd-bef3-49e6-aae8-c5a7ca681f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:00.947138 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe52b0-8b11-4bd4-8084-0cf3d8524567 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:00.948623 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe52b0-8b11-4bd4-8084-0cf3d8524567 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-91fe52b0-8b11-4bd4-8084-0cf3d8524567 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:37:00.963312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:00.964324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:00.965267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:00.965267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:00.965267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:37:00.965267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:37:00.965267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:37:00.965267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:37:00.965267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:37:00.965267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:37:00.965267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 03:37:00.965267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:00.965267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 468/927] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 12 03:37:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:00.966298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a4e6-9449-450d-b11d-fc6cbc403875 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:00.966407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a4e6-9449-450d-b11d-fc6cbc403875 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:00.966652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922a4e6-9449-450d-b11d-fc6cbc403875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:00.988202 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-1922a4e6-9449-450d-b11d-fc6cbc403875 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:00.989346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1922a4e6-9449-450d-b11d-fc6cbc403875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:01.016620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 460/928] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:01.017545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fe2dd-bef3-49e6-aae8-c5a7ca681f2c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:01.077409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 469/929] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:01.079224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922a4e6-9449-450d-b11d-fc6cbc403875 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:01.107908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:01.116651 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:01.121757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fe2dd-bef3-49e6-aae8-c5a7ca681f2c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:01.159435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531de33-1a23-41e9-8859-bade3c38f836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:01.160130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531de33-1a23-41e9-8859-bade3c38f836 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:01.160411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531de33-1a23-41e9-8859-bade3c38f836 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:01.160689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531de33-1a23-41e9-8859-bade3c38f836 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:01.161528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531de33-1a23-41e9-8859-bade3c38f836 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:01.161932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531de33-1a23-41e9-8859-bade3c38f836 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:01.163305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531de33-1a23-41e9-8859-bade3c38f836 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:01.186775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 461/930] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Jul 12 03:37:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:01.187605 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531de33-1a23-41e9-8859-bade3c38f836 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:01.202109 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922a4e6-9449-450d-b11d-fc6cbc403875 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:01.239832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338de5e7-872b-404c-8191-9d50d331d3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:01.240654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338de5e7-872b-404c-8191-9d50d331d3ac None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:01.240944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338de5e7-872b-404c-8191-9d50d331d3ac None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:01.241220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338de5e7-872b-404c-8191-9d50d331d3ac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:01.241873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338de5e7-872b-404c-8191-9d50d331d3ac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:01.243255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338de5e7-872b-404c-8191-9d50d331d3ac None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-338de5e7-872b-404c-8191-9d50d331d3ac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:01.255512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:01.256501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:01.257344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 470/931] 10.222.0.20 () {56 vars in 1132 bytes} [Tue Jul 12 03:37:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:01.258204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338de5e7-872b-404c-8191-9d50d331d3ac None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:01.311052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531de33-1a23-41e9-8859-bade3c38f836 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:01.349531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e13d3-dd27-4f1b-852c-5fd8fc05cff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:01.350342 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e13d3-dd27-4f1b-852c-5fd8fc05cff1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:01.350623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e13d3-dd27-4f1b-852c-5fd8fc05cff1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:01.350898 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e13d3-dd27-4f1b-852c-5fd8fc05cff1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:01.351546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e13d3-dd27-4f1b-852c-5fd8fc05cff1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:01.352952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e13d3-dd27-4f1b-852c-5fd8fc05cff1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:01.355260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:01.363733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338de5e7-872b-404c-8191-9d50d331d3ac None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6f7e13d3-dd27-4f1b-852c-5fd8fc05cff1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:01.365200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:01.366154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:37:01.366967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:01.371768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 462/932] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 12 03:37:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:01.372533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e13d3-dd27-4f1b-852c-5fd8fc05cff1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:01.385020 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:01.405552 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7f0c9-60f4-4d07-a68c-3761ff885455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:01.406645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7f0c9-60f4-4d07-a68c-3761ff885455 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:01.407081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7f0c9-60f4-4d07-a68c-3761ff885455 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:01.407555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7f0c9-60f4-4d07-a68c-3761ff885455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:01.419067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c7f0c9-60f4-4d07-a68c-3761ff885455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:01.420599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c7f0c9-60f4-4d07-a68c-3761ff885455 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-b7c7f0c9-60f4-4d07-a68c-3761ff885455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:37:01.440306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:01.441368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:01.443133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:37:01.443133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:37:01.443133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:01.443133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:01.443133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 471/933] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:01.443133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7f0c9-60f4-4d07-a68c-3761ff885455 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:01.487747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e13d3-dd27-4f1b-852c-5fd8fc05cff1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:01.522864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942701e-c4be-4160-9fc9-d623ae9a403c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:01.523444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942701e-c4be-4160-9fc9-d623ae9a403c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:01.523723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942701e-c4be-4160-9fc9-d623ae9a403c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:01.523996 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942701e-c4be-4160-9fc9-d623ae9a403c None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:01.524733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942701e-c4be-4160-9fc9-d623ae9a403c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:01.525037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942701e-c4be-4160-9fc9-d623ae9a403c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:01.526408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5942701e-c4be-4160-9fc9-d623ae9a403c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:01.531684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 463/934] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 12 03:37:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:01.532484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942701e-c4be-4160-9fc9-d623ae9a403c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:01.555072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7f0c9-60f4-4d07-a68c-3761ff885455 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:01.590459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1da99-186c-4cf8-ae13-e228f83afbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:01.591165 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1da99-186c-4cf8-ae13-e228f83afbfa None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:01.591450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1da99-186c-4cf8-ae13-e228f83afbfa None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:01.591759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1da99-186c-4cf8-ae13-e228f83afbfa None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:01.592516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1da99-186c-4cf8-ae13-e228f83afbfa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:01.592806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1da99-186c-4cf8-ae13-e228f83afbfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:01.594185 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1da99-186c-4cf8-ae13-e228f83afbfa None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:01.599081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 472/935] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 12 03:37:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:01.599878 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1da99-186c-4cf8-ae13-e228f83afbfa None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:01.615039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:01.618048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:01.643521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942701e-c4be-4160-9fc9-d623ae9a403c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:01.679589 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072255b7-6d17-40c5-9477-2788fd0ae4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:01.680273 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072255b7-6d17-40c5-9477-2788fd0ae4f1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:01.680560 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072255b7-6d17-40c5-9477-2788fd0ae4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:01.680833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072255b7-6d17-40c5-9477-2788fd0ae4f1 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:01.681540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072255b7-6d17-40c5-9477-2788fd0ae4f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:01.681841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072255b7-6d17-40c5-9477-2788fd0ae4f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:01.683209 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072255b7-6d17-40c5-9477-2788fd0ae4f1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:01.699725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 464/936] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:01.699725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072255b7-6d17-40c5-9477-2788fd0ae4f1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:01.719208 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1da99-186c-4cf8-ae13-e228f83afbfa None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:01.758426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8bb98-7272-45a0-801a-12c689aba5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:01.759531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8bb98-7272-45a0-801a-12c689aba5e7 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:01.760117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8bb98-7272-45a0-801a-12c689aba5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:01.760895 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8bb98-7272-45a0-801a-12c689aba5e7 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:01.761753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8bb98-7272-45a0-801a-12c689aba5e7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:01.762320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8bb98-7272-45a0-801a-12c689aba5e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:01.763930 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8bb98-7272-45a0-801a-12c689aba5e7 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:01.769875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 473/937] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 12 03:37:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:01.770771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8bb98-7272-45a0-801a-12c689aba5e7 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:01.808248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072255b7-6d17-40c5-9477-2788fd0ae4f1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:01.842481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d5574-8ffb-4602-9540-c3c112b9a9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:01.843147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d5574-8ffb-4602-9540-c3c112b9a9cb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:01.843433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d5574-8ffb-4602-9540-c3c112b9a9cb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:01.843709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d5574-8ffb-4602-9540-c3c112b9a9cb None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:01.844520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d5574-8ffb-4602-9540-c3c112b9a9cb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:01.844832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d5574-8ffb-4602-9540-c3c112b9a9cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:01.846210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d5574-8ffb-4602-9540-c3c112b9a9cb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:01.847448 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:01.850611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:01.851996 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 465/938] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 12 03:37:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:01.857633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d5574-8ffb-4602-9540-c3c112b9a9cb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:01.878250 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8bb98-7272-45a0-801a-12c689aba5e7 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:01.914226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5cfa7-f4ac-4d5e-8af7-36bb066b6ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:01.915349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5cfa7-f4ac-4d5e-8af7-36bb066b6ecb None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:01.915822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5cfa7-f4ac-4d5e-8af7-36bb066b6ecb None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:01.916132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5cfa7-f4ac-4d5e-8af7-36bb066b6ecb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:01.922873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5cfa7-f4ac-4d5e-8af7-36bb066b6ecb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:01.924354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5cfa7-f4ac-4d5e-8af7-36bb066b6ecb None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-ead5cfa7-f4ac-4d5e-8af7-36bb066b6ecb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:37:01.939471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:01.940431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:01.941319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:37:01.941319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:37:01.941319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:37:01.941319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:37:01.941319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:37:01.941319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:37:01.941319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:37:01.941319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:01.945651 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 474/939] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:37:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:01.945651 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5cfa7-f4ac-4d5e-8af7-36bb066b6ecb None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:01.968779 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d5574-8ffb-4602-9540-c3c112b9a9cb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:02.007634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e6232-4f80-4de9-b7ab-d4e1a809ca83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:02.008335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e6232-4f80-4de9-b7ab-d4e1a809ca83 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:02.008620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e6232-4f80-4de9-b7ab-d4e1a809ca83 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:02.008900 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e6232-4f80-4de9-b7ab-d4e1a809ca83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:02.009639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e6232-4f80-4de9-b7ab-d4e1a809ca83 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:02.011345 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e6232-4f80-4de9-b7ab-d4e1a809ca83 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:02.031687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 466/940] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 12 03:37:01 2022] POST /identity/v3/endpoints => generated 347 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:37:02.032539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e6232-4f80-4de9-b7ab-d4e1a809ca83 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:02.065418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5cfa7-f4ac-4d5e-8af7-36bb066b6ecb None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:02.101902 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:02.122922 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:02.130209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3d5cc-8611-453e-9f3d-0585b9abf225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:02.130884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3d5cc-8611-453e-9f3d-0585b9abf225 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:02.131221 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3d5cc-8611-453e-9f3d-0585b9abf225 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:02.131503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3d5cc-8611-453e-9f3d-0585b9abf225 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:02.132353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3d5cc-8611-453e-9f3d-0585b9abf225 None admin] RBAC: Adding query filter params (interface=public, service_id=0ebb232015f24127a1a33120769eaef2, region_id=RegionOne) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:02.132670 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3d5cc-8611-453e-9f3d-0585b9abf225 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:02.138468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3d5cc-8611-453e-9f3d-0585b9abf225 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:02.149159 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 475/941] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 12 03:37:01 2022] GET /identity/v3/endpoints?service_id=0ebb232015f24127a1a33120769eaef2&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:02.150139 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3d5cc-8611-453e-9f3d-0585b9abf225 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:02.163689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e6232-4f80-4de9-b7ab-d4e1a809ca83 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:02.203410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26547a46-ca9e-4697-ac9c-2ff97c8c83e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:02.204203 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547a46-ca9e-4697-ac9c-2ff97c8c83e9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:02.204494 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547a46-ca9e-4697-ac9c-2ff97c8c83e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:02.204772 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547a46-ca9e-4697-ac9c-2ff97c8c83e9 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:02.205572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26547a46-ca9e-4697-ac9c-2ff97c8c83e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:02.205885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26547a46-ca9e-4697-ac9c-2ff97c8c83e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:02.207359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26547a46-ca9e-4697-ac9c-2ff97c8c83e9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:02.227547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 467/942] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:37:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:02.234239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26547a46-ca9e-4697-ac9c-2ff97c8c83e9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:02.296424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3d5cc-8611-453e-9f3d-0585b9abf225 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:02.331405 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5b6ab-f123-421b-b6cf-5c60c77368ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:02.332420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5b6ab-f123-421b-b6cf-5c60c77368ac None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:02.332990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5b6ab-f123-421b-b6cf-5c60c77368ac None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:02.333444 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5b6ab-f123-421b-b6cf-5c60c77368ac None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:02.334409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5b6ab-f123-421b-b6cf-5c60c77368ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:02.335974 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5b6ab-f123-421b-b6cf-5c60c77368ac None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:02.343142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 476/943] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 12 03:37:02 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:37:02.357664 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26547a46-ca9e-4697-ac9c-2ff97c8c83e9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:02.374158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:02.396722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832043ed-33a3-44af-8e54-1caa15870230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:02.397648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832043ed-33a3-44af-8e54-1caa15870230 None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:02.397935 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832043ed-33a3-44af-8e54-1caa15870230 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:02.398209 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832043ed-33a3-44af-8e54-1caa15870230 None admin] PATH_INFO: `/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/2b01c3b5c3384d4ca0999e9156c3a9cb/roles/eea62fc4e0174ba39b800050ebe7ff5d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:02.402001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832043ed-33a3-44af-8e54-1caa15870230 None admin] RBAC: Authorizing `identity:create_grant(project_id=48b17053daa84e4fa94840777cb1bee6, user_id=2b01c3b5c3384d4ca0999e9156c3a9cb, role_id=eea62fc4e0174ba39b800050ebe7ff5d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:02.403568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832043ed-33a3-44af-8e54-1caa15870230 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:02.421217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 468/944] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:37:02 2022] PUT /identity/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/2b01c3b5c3384d4ca0999e9156c3a9cb/roles/eea62fc4e0174ba39b800050ebe7ff5d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:37:02.560996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08714e-06cf-4042-873d-97e89c19db2f None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:02.561383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08714e-06cf-4042-873d-97e89c19db2f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:02.561599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08714e-06cf-4042-873d-97e89c19db2f None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:02.563344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 477/945] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:02.569327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9d788-ca9c-49c2-8145-cf636a4b7565 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:02.569662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9d788-ca9c-49c2-8145-cf636a4b7565 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:02.569890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9d788-ca9c-49c2-8145-cf636a4b7565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:02.588340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-40f9d788-ca9c-49c2-8145-cf636a4b7565 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:02.589453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-40f9d788-ca9c-49c2-8145-cf636a4b7565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:02.668085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 469/946] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:02.684323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08714e-06cf-4042-873d-97e89c19db2f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:02.744350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:02.791853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08714e-06cf-4042-873d-97e89c19db2f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:02.826718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b24dd-5623-43ac-9f5b-81e5df863472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:02.827381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b24dd-5623-43ac-9f5b-81e5df863472 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:02.827666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b24dd-5623-43ac-9f5b-81e5df863472 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:02.827941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b24dd-5623-43ac-9f5b-81e5df863472 None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:02.828682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b24dd-5623-43ac-9f5b-81e5df863472 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:02.830117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b24dd-5623-43ac-9f5b-81e5df863472 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:02.831486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: WARNING keystone.server.flask.common [None req-a83b24dd-5623-43ac-9f5b-81e5df863472 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 12 03:37:02.849995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.sql.core [None req-a83b24dd-5623-43ac-9f5b-81e5df863472 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 03:37:02.849995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 03:37:02.849995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [parameters: {'id': '1317f57dc6a7407fb3932faff8b01cf5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 03:37:02.849995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86935) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-a83b24dd-5623-43ac-9f5b-81e5df863472 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1317f57dc6a7407fb3932faff8b01cf5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1317f57dc6a7407fb3932faff8b01cf5. Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application conn.query(q) Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 03:37:02.857666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result.read() Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 12 03:37:02.858467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application next(self.gen) Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application yield resource Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application next(self.gen) Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise self.value Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application session.commit() Jul 12 03:37:02.859395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.session.flush() Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application compat.raise_( Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 03:37:02.860320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 03:37:02.861232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application conn.query(q) Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result.read() Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 03:37:02.862029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [parameters: {'id': '1317f57dc6a7407fb3932faff8b01cf5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:02.862892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:02.863656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:02.870666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:02.870666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 12 03:37:02.870666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 12 03:37:02.870666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:02.870666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:02.870666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 12 03:37:02.870666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 12 03:37:02.870666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1317f57dc6a7407fb3932faff8b01cf5. Jul 12 03:37:02.870666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:02.870666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 478/947] 10.222.0.20 () {60 vars in 1140 bytes} [Tue Jul 12 03:37:02 2022] POST /identity/v3/projects => generated 262 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:02.871934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9d788-ca9c-49c2-8145-cf636a4b7565 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:02.981105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9d788-ca9c-49c2-8145-cf636a4b7565 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:02.983637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:03.020108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b7a27-5978-4dfe-a5fa-294b3cdde87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:03.020912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b7a27-5978-4dfe-a5fa-294b3cdde87d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:03.021202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b7a27-5978-4dfe-a5fa-294b3cdde87d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:03.021472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b7a27-5978-4dfe-a5fa-294b3cdde87d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:03.028735 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b7a27-5978-4dfe-a5fa-294b3cdde87d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:03.030323 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b7a27-5978-4dfe-a5fa-294b3cdde87d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-0c0b7a27-5978-4dfe-a5fa-294b3cdde87d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:37:03.044371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:03.045318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:03.046439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:37:03.046439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:37:03.046439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:37:03.046439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:37:03.046439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:37:03.046439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:37:03.046439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:37:03.046439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:03.050425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 470/948] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:37:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:03.052095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b24dd-5623-43ac-9f5b-81e5df863472 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:03.156830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b24dd-5623-43ac-9f5b-81e5df863472 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:03.171631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:03.191950 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5e2b4-da1c-4c8d-bc03-bfb00c2bfddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:03.192634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5e2b4-da1c-4c8d-bc03-bfb00c2bfddd None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:03.192918 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5e2b4-da1c-4c8d-bc03-bfb00c2bfddd None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:03.193200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5e2b4-da1c-4c8d-bc03-bfb00c2bfddd None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:03.193955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5e2b4-da1c-4c8d-bc03-bfb00c2bfddd None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:03.194261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5e2b4-da1c-4c8d-bc03-bfb00c2bfddd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:03.195898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5e2b4-da1c-4c8d-bc03-bfb00c2bfddd None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:03.216246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 479/949] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:37:03 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:04.195275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370cf88-c57b-49d5-af24-0df29ea57261 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:04.196307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370cf88-c57b-49d5-af24-0df29ea57261 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:04.196806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370cf88-c57b-49d5-af24-0df29ea57261 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:04.198583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 471/950] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:04.204624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9ab9a-57aa-4e46-a31d-f5b8a98f704e None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:04.205487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9ab9a-57aa-4e46-a31d-f5b8a98f704e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:04.205961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9ab9a-57aa-4e46-a31d-f5b8a98f704e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:04.224642 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-43d9ab9a-57aa-4e46-a31d-f5b8a98f704e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:04.226475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-43d9ab9a-57aa-4e46-a31d-f5b8a98f704e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:04.295625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 480/951] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:04.318114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370cf88-c57b-49d5-af24-0df29ea57261 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:04.427177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370cf88-c57b-49d5-af24-0df29ea57261 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:04.430761 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fcc27-f35c-4cd4-837f-c5be8f1f1d35 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:04.431055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fcc27-f35c-4cd4-837f-c5be8f1f1d35 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:04.431336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fcc27-f35c-4cd4-837f-c5be8f1f1d35 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:04.433005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 481/952] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:37:04.438021 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b3b53-6555-4d05-ae4f-123f8840bfd3 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:04.438312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b3b53-6555-4d05-ae4f-123f8840bfd3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:04.438594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b3b53-6555-4d05-ae4f-123f8840bfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:04.458125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-0a7b3b53-6555-4d05-ae4f-123f8840bfd3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:04.459459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0a7b3b53-6555-4d05-ae4f-123f8840bfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:04.465527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fbf43-65b9-4c5e-8cc5-6344300ba31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:04.466684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fbf43-65b9-4c5e-8cc5-6344300ba31d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:04.467172 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fbf43-65b9-4c5e-8cc5-6344300ba31d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:04.467657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fbf43-65b9-4c5e-8cc5-6344300ba31d None admin] PATH_INFO: `/v3/services/compute` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:04.468654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fbf43-65b9-4c5e-8cc5-6344300ba31d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:04.470346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fbf43-65b9-4c5e-8cc5-6344300ba31d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a87fbf43-65b9-4c5e-8cc5-6344300ba31d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:04.483741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:04.484889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 03:37:04.485806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:04.488745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 472/953] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:37:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:04.494563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fbf43-65b9-4c5e-8cc5-6344300ba31d None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:04.533332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:04.539052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 482/954] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:37:04.563682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b3b53-6555-4d05-ae4f-123f8840bfd3 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:04.600355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:04.616736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fbf43-65b9-4c5e-8cc5-6344300ba31d None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:04.652135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663a829-9442-4f2d-94e8-24f891bf5ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:04.652803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663a829-9442-4f2d-94e8-24f891bf5ccd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:04.653098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663a829-9442-4f2d-94e8-24f891bf5ccd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:04.653493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663a829-9442-4f2d-94e8-24f891bf5ccd None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:04.654078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663a829-9442-4f2d-94e8-24f891bf5ccd None admin] RBAC: Adding query filter params (name=compute) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:04.654389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663a829-9442-4f2d-94e8-24f891bf5ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:04.655764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663a829-9442-4f2d-94e8-24f891bf5ccd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:04.661155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 473/955] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:37:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:04.662187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25e150-9ec1-4296-8ae1-df5c2c7cd353 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:04.662554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25e150-9ec1-4296-8ae1-df5c2c7cd353 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:04.662836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25e150-9ec1-4296-8ae1-df5c2c7cd353 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:04.664180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 474/956] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:04.668692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25e150-9ec1-4296-8ae1-df5c2c7cd353 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:04.673175 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b3b53-6555-4d05-ae4f-123f8840bfd3 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:04.708454 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a8da5-016b-4fba-bf0d-e011f20e9abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:04.709268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a8da5-016b-4fba-bf0d-e011f20e9abd None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:04.709551 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a8da5-016b-4fba-bf0d-e011f20e9abd None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:04.709824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a8da5-016b-4fba-bf0d-e011f20e9abd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:04.710489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a8da5-016b-4fba-bf0d-e011f20e9abd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:04.711888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a8da5-016b-4fba-bf0d-e011f20e9abd None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-857a8da5-016b-4fba-bf0d-e011f20e9abd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:04.723947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:04.725168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:04.726008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 483/957] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 12 03:37:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:04.726804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fdf28-afd5-4aa2-b958-831cadfbc5e2 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:04.726804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fdf28-afd5-4aa2-b958-831cadfbc5e2 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:04.726929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fdf28-afd5-4aa2-b958-831cadfbc5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:04.744097 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-055fdf28-afd5-4aa2-b958-831cadfbc5e2 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:04.745204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-055fdf28-afd5-4aa2-b958-831cadfbc5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:04.782820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:04.792781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25e150-9ec1-4296-8ae1-df5c2c7cd353 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:04.814555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 484/958] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:04.815469 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055fdf28-afd5-4aa2-b958-831cadfbc5e2 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:04.841591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383f901-6527-4bfc-8681-41d797eaae57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:04.842467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383f901-6527-4bfc-8681-41d797eaae57 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:04.842755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383f901-6527-4bfc-8681-41d797eaae57 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:04.843037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383f901-6527-4bfc-8681-41d797eaae57 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:04.843760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383f901-6527-4bfc-8681-41d797eaae57 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:04.844080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383f901-6527-4bfc-8681-41d797eaae57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:04.844693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:04.845460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383f901-6527-4bfc-8681-41d797eaae57 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:04.850714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 475/959] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:37:04 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:04.851531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383f901-6527-4bfc-8681-41d797eaae57 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:04.938900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055fdf28-afd5-4aa2-b958-831cadfbc5e2 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:04.983135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79144e-b965-4cfc-ab29-85b9c653ce30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:04.983879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79144e-b965-4cfc-ab29-85b9c653ce30 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:04.984197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79144e-b965-4cfc-ab29-85b9c653ce30 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:04.984478 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79144e-b965-4cfc-ab29-85b9c653ce30 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:04.985259 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79144e-b965-4cfc-ab29-85b9c653ce30 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:04.985570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79144e-b965-4cfc-ab29-85b9c653ce30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:04.987773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79144e-b965-4cfc-ab29-85b9c653ce30 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:04.996416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 485/960] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 12 03:37:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:05.001041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383f901-6527-4bfc-8681-41d797eaae57 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:05.001922 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79144e-b965-4cfc-ab29-85b9c653ce30 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:05.041067 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df765ab-ac74-4f89-8494-72bb05303dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:05.042854 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df765ab-ac74-4f89-8494-72bb05303dd1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:05.043243 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df765ab-ac74-4f89-8494-72bb05303dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:05.043618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df765ab-ac74-4f89-8494-72bb05303dd1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:05.055272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df765ab-ac74-4f89-8494-72bb05303dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6df765ab-ac74-4f89-8494-72bb05303dd1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:37:05.064921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:05.065908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:05.066744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 476/961] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Jul 12 03:37:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:05.072648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df765ab-ac74-4f89-8494-72bb05303dd1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:05.093908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:05.116730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:05.128605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79144e-b965-4cfc-ab29-85b9c653ce30 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:05.170661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661ac79-ed5f-447e-93b6-c623118c889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:05.171653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661ac79-ed5f-447e-93b6-c623118c889c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:05.172278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661ac79-ed5f-447e-93b6-c623118c889c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:05.172703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661ac79-ed5f-447e-93b6-c623118c889c None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:05.173658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661ac79-ed5f-447e-93b6-c623118c889c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:05.174110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661ac79-ed5f-447e-93b6-c623118c889c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:05.175658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661ac79-ed5f-447e-93b6-c623118c889c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:05.186207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df765ab-ac74-4f89-8494-72bb05303dd1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:05.186682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 486/962] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 12 03:37:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:05.187769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661ac79-ed5f-447e-93b6-c623118c889c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:05.221598 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abe1a5-846b-4f06-b1e4-bf45f7a21829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:05.222413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abe1a5-846b-4f06-b1e4-bf45f7a21829 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:05.222758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abe1a5-846b-4f06-b1e4-bf45f7a21829 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:05.223091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abe1a5-846b-4f06-b1e4-bf45f7a21829 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:05.223905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79abe1a5-846b-4f06-b1e4-bf45f7a21829 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:05.224299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79abe1a5-846b-4f06-b1e4-bf45f7a21829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:05.225773 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79abe1a5-846b-4f06-b1e4-bf45f7a21829 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:05.239677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 477/963] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 12 03:37:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:05.240435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abe1a5-846b-4f06-b1e4-bf45f7a21829 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:05.302122 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661ac79-ed5f-447e-93b6-c623118c889c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:05.323423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:05.338925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f83a58-43ae-438e-89b0-9a15694b051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:05.339649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f83a58-43ae-438e-89b0-9a15694b051f None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:05.339938 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f83a58-43ae-438e-89b0-9a15694b051f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:05.340241 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f83a58-43ae-438e-89b0-9a15694b051f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:05.341024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f83a58-43ae-438e-89b0-9a15694b051f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:05.342436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f83a58-43ae-438e-89b0-9a15694b051f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:05.345415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abe1a5-846b-4f06-b1e4-bf45f7a21829 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:05.373852 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 487/964] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 12 03:37:05 2022] POST /identity/v3/endpoints => generated 349 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:37:05.374433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3556d-9dee-40df-8432-bfda9f08071a None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:05.374831 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3556d-9dee-40df-8432-bfda9f08071a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:05.375118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3556d-9dee-40df-8432-bfda9f08071a None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:05.381983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 488/965] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:05.385313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11ce06-9d41-4a71-8cbd-4f86c5ec9a55 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:05.385821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11ce06-9d41-4a71-8cbd-4f86c5ec9a55 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:05.386268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11ce06-9d41-4a71-8cbd-4f86c5ec9a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:05.401757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200903f-743b-4a24-966a-7c725dd582e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:05.402696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200903f-743b-4a24-966a-7c725dd582e2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:05.403044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200903f-743b-4a24-966a-7c725dd582e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:05.403443 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200903f-743b-4a24-966a-7c725dd582e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:05.408778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-3b11ce06-9d41-4a71-8cbd-4f86c5ec9a55 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:05.410374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3b11ce06-9d41-4a71-8cbd-4f86c5ec9a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:05.417056 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200903f-743b-4a24-966a-7c725dd582e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:05.418649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200903f-743b-4a24-966a-7c725dd582e2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:05.421580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6200903f-743b-4a24-966a-7c725dd582e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:37:05.442762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:05.443776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:05.444813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:37:05.444813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:37:05.444813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:05.444813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:05.449151 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 478/966] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:05.450376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200903f-743b-4a24-966a-7c725dd582e2 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:05.498889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 489/967] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:05.530353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11ce06-9d41-4a71-8cbd-4f86c5ec9a55 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:05.572081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200903f-743b-4a24-966a-7c725dd582e2 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:05.601431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:05.620163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474246c-8075-4965-86ef-7948dacff9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:05.621688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474246c-8075-4965-86ef-7948dacff9ff None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:05.622188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474246c-8075-4965-86ef-7948dacff9ff None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:05.623107 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474246c-8075-4965-86ef-7948dacff9ff None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:05.624436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474246c-8075-4965-86ef-7948dacff9ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:05.624951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474246c-8075-4965-86ef-7948dacff9ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:05.627583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4474246c-8075-4965-86ef-7948dacff9ff None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:05.646287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11ce06-9d41-4a71-8cbd-4f86c5ec9a55 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:05.647024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 479/968] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:05.655887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474246c-8075-4965-86ef-7948dacff9ff None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:05.682236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2ec62-6da4-444e-bcd5-816e3352ae5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:05.682909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ec62-6da4-444e-bcd5-816e3352ae5c None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:05.683193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ec62-6da4-444e-bcd5-816e3352ae5c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:05.683541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ec62-6da4-444e-bcd5-816e3352ae5c None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:05.684186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2ec62-6da4-444e-bcd5-816e3352ae5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:05.685912 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2ec62-6da4-444e-bcd5-816e3352ae5c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:05.700258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.sql.core [None req-0eb2ec62-6da4-444e-bcd5-816e3352ae5c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 03:37:05.700258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 03:37:05.700258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [parameters: {'user_id': '2edeeb23b1c6417e9706a726df19e642', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 03:37:05.700258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86935) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-0eb2ec62-6da4-444e-bcd5-816e3352ae5c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application conn.query(q) Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result.read() Jul 12 03:37:05.708417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 03:37:05.709269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application next(self.gen) Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application yield resource Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application next(self.gen) Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise self.value Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application session.commit() Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 03:37:05.710719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.session.flush() Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application compat.raise_( Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 03:37:05.711823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 03:37:05.712741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application conn.query(q) Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application result.read() Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 03:37:05.713579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [parameters: {'user_id': '2edeeb23b1c6417e9706a726df19e642', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:05.714406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 12 03:37:05.717014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 12 03:37:05.729280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 12 03:37:05.729280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 12 03:37:05.729280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 03:37:05.729280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 03:37:05.729280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 03:37:05.729280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:05.729280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 490/969] 10.222.0.20 () {60 vars in 1131 bytes} [Tue Jul 12 03:37:05 2022] POST /identity/v3/users => generated 160 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:37:05.729280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2ec62-6da4-444e-bcd5-816e3352ae5c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:05.730840 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:05.759313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474246c-8075-4965-86ef-7948dacff9ff None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:05.795804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90792e3c-733d-4dc8-8673-6f339de3aa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:05.796656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90792e3c-733d-4dc8-8673-6f339de3aa03 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:05.796940 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90792e3c-733d-4dc8-8673-6f339de3aa03 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:05.797212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90792e3c-733d-4dc8-8673-6f339de3aa03 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:05.811438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90792e3c-733d-4dc8-8673-6f339de3aa03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:05.813604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90792e3c-733d-4dc8-8673-6f339de3aa03 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:05.830927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2ec62-6da4-444e-bcd5-816e3352ae5c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-90792e3c-733d-4dc8-8673-6f339de3aa03 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:37:05.832610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:05.833524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:05.834476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:05.834476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:05.834476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:37:05.834476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:37:05.834476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:37:05.834476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:37:05.834476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:37:05.834476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:37:05.834476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 03:37:05.834476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:05.835180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 480/970] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 12 03:37:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:05.840857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90792e3c-733d-4dc8-8673-6f339de3aa03 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:05.873439 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36519843-3f89-4eb6-a143-7746ec0a9972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:05.874481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36519843-3f89-4eb6-a143-7746ec0a9972 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:05.874863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36519843-3f89-4eb6-a143-7746ec0a9972 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:05.875257 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36519843-3f89-4eb6-a143-7746ec0a9972 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:05.883921 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36519843-3f89-4eb6-a143-7746ec0a9972 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:05.886254 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36519843-3f89-4eb6-a143-7746ec0a9972 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:05.897852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-36519843-3f89-4eb6-a143-7746ec0a9972 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:37:05.908614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:05.909754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:05.910999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:05.910999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:05.910999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:37:05.910999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:37:05.910999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:37:05.910999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:37:05.910999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:37:05.910999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:37:05.910999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 03:37:05.910999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:05.913089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 491/971] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:37:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:37:05.920035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36519843-3f89-4eb6-a143-7746ec0a9972 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:05.972174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90792e3c-733d-4dc8-8673-6f339de3aa03 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:06.000104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:06.015403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6186c-4b10-4bb0-b25e-350bc6b94cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:06.016188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6186c-4b10-4bb0-b25e-350bc6b94cdc None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:06.017066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6186c-4b10-4bb0-b25e-350bc6b94cdc None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:06.017350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6186c-4b10-4bb0-b25e-350bc6b94cdc None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:06.018237 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6186c-4b10-4bb0-b25e-350bc6b94cdc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:06.018549 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6186c-4b10-4bb0-b25e-350bc6b94cdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:06.020593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6186c-4b10-4bb0-b25e-350bc6b94cdc None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:06.043915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 481/972] 10.222.0.20 () {56 vars in 1149 bytes} [Tue Jul 12 03:37:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:06.049656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6186c-4b10-4bb0-b25e-350bc6b94cdc None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:06.053931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36519843-3f89-4eb6-a143-7746ec0a9972 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:06.091501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c53c69-fe2c-4c1d-b6c9-6cbe481a7bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:06.092131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c53c69-fe2c-4c1d-b6c9-6cbe481a7bab None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:06.092508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c53c69-fe2c-4c1d-b6c9-6cbe481a7bab None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:06.092790 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c53c69-fe2c-4c1d-b6c9-6cbe481a7bab None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:06.093637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c53c69-fe2c-4c1d-b6c9-6cbe481a7bab None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:06.093964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c53c69-fe2c-4c1d-b6c9-6cbe481a7bab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:06.096232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c53c69-fe2c-4c1d-b6c9-6cbe481a7bab None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:06.143633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 492/973] 10.222.0.20 () {56 vars in 1101 bytes} [Tue Jul 12 03:37:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:06.171428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6186c-4b10-4bb0-b25e-350bc6b94cdc None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:06.174144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:06.210408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1373f-a863-4c9e-bfed-8938759a7a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:06.211149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1373f-a863-4c9e-bfed-8938759a7a60 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:06.211434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1373f-a863-4c9e-bfed-8938759a7a60 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:06.211708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb1373f-a863-4c9e-bfed-8938759a7a60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:06.291818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1373f-a863-4c9e-bfed-8938759a7a60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:06.294162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb1373f-a863-4c9e-bfed-8938759a7a60 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:06.401250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 482/974] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:37:06 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 03:37:06.407464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c53c69-fe2c-4c1d-b6c9-6cbe481a7bab None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:06.518245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c53c69-fe2c-4c1d-b6c9-6cbe481a7bab None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:06.542402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:06.555486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070b9b7-8801-42b7-b444-4f7f16b881d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:06.556348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070b9b7-8801-42b7-b444-4f7f16b881d7 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:06.556626 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070b9b7-8801-42b7-b444-4f7f16b881d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:06.556897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070b9b7-8801-42b7-b444-4f7f16b881d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:06.567794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070b9b7-8801-42b7-b444-4f7f16b881d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:06.569282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070b9b7-8801-42b7-b444-4f7f16b881d7 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-b070b9b7-8801-42b7-b444-4f7f16b881d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:37:06.586909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:06.587818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:06.589051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:37:06.589051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:37:06.589051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:06.589051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:06.592093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 493/975] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:06.594071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1373f-a863-4c9e-bfed-8938759a7a60 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:06.597568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:06.704518 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb1373f-a863-4c9e-bfed-8938759a7a60 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:06.738882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ef08-6e71-4d28-9a12-1915e0977910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:06.739718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ef08-6e71-4d28-9a12-1915e0977910 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:06.740157 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ef08-6e71-4d28-9a12-1915e0977910 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:06.740527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8ef08-6e71-4d28-9a12-1915e0977910 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:06.741352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8ef08-6e71-4d28-9a12-1915e0977910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:06.741751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8ef08-6e71-4d28-9a12-1915e0977910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:06.743237 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8ef08-6e71-4d28-9a12-1915e0977910 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:06.759160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 483/976] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:06.760915 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070b9b7-8801-42b7-b444-4f7f16b881d7 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:06.860590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070b9b7-8801-42b7-b444-4f7f16b881d7 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:06.894028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52ed02-46fb-496f-9be0-5d57a709357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:06.894957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52ed02-46fb-496f-9be0-5d57a709357a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:06.895340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52ed02-46fb-496f-9be0-5d57a709357a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:06.895710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52ed02-46fb-496f-9be0-5d57a709357a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:06.902563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52ed02-46fb-496f-9be0-5d57a709357a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:06.904179 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52ed02-46fb-496f-9be0-5d57a709357a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-de52ed02-46fb-496f-9be0-5d57a709357a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:37:06.918443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:06.919571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:06.920695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:37:06.920695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:37:06.920695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:37:06.920695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:37:06.920695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:37:06.920695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:37:06.920695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:37:06.920695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:06.922334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 494/977] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:37:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:06.927988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ef08-6e71-4d28-9a12-1915e0977910 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:06.960291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f48fd-1daa-4dfb-a659-55caee34a4f7 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:06.960474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f48fd-1daa-4dfb-a659-55caee34a4f7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:06.960765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f48fd-1daa-4dfb-a659-55caee34a4f7 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:06.962306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 495/978] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:06.967056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379a20b-9dfd-40a5-8793-a9b84f2eaf60 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:06.967352 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379a20b-9dfd-40a5-8793-a9b84f2eaf60 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:06.967661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379a20b-9dfd-40a5-8793-a9b84f2eaf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:06.984932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-0379a20b-9dfd-40a5-8793-a9b84f2eaf60 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:06.986459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0379a20b-9dfd-40a5-8793-a9b84f2eaf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:07.001736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:07.015406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:07.049111 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8ef08-6e71-4d28-9a12-1915e0977910 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:07.056032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 496/979] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:37:07.078500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379a20b-9dfd-40a5-8793-a9b84f2eaf60 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:07.085664 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3a899-0ab2-4de6-b64a-cfe946da162b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:07.086336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3a899-0ab2-4de6-b64a-cfe946da162b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:07.087048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3a899-0ab2-4de6-b64a-cfe946da162b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:07.087326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3a899-0ab2-4de6-b64a-cfe946da162b None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:07.088161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3a899-0ab2-4de6-b64a-cfe946da162b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:07.088817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3a899-0ab2-4de6-b64a-cfe946da162b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:07.090745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3a899-0ab2-4de6-b64a-cfe946da162b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:07.105623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 484/980] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:37:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:07.111581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3a899-0ab2-4de6-b64a-cfe946da162b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:07.186945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379a20b-9dfd-40a5-8793-a9b84f2eaf60 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:07.215932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3a899-0ab2-4de6-b64a-cfe946da162b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:07.223615 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5f2b9-27d8-49b0-bfe8-48e1f821a009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:07.224334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5f2b9-27d8-49b0-bfe8-48e1f821a009 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:07.224621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5f2b9-27d8-49b0-bfe8-48e1f821a009 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:07.224902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5f2b9-27d8-49b0-bfe8-48e1f821a009 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:07.225625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5f2b9-27d8-49b0-bfe8-48e1f821a009 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:07.227006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5f2b9-27d8-49b0-bfe8-48e1f821a009 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:07.242598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 497/981] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:37:07 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:37:07.243543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:07.263638 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a392da-4405-49d8-9307-71ca059327eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:07.264363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a392da-4405-49d8-9307-71ca059327eb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:07.264651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a392da-4405-49d8-9307-71ca059327eb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:07.264919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a392da-4405-49d8-9307-71ca059327eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:07.265779 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a392da-4405-49d8-9307-71ca059327eb None admin] RBAC: Adding query filter params (role.id=eea62fc4e0174ba39b800050ebe7ff5d, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=2b01c3b5c3384d4ca0999e9156c3a9cb) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:07.266099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a392da-4405-49d8-9307-71ca059327eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:07.267674 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a392da-4405-49d8-9307-71ca059327eb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:07.280611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 485/982] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:37:07 2022] GET /identity/v3/role_assignments?user.id=2b01c3b5c3384d4ca0999e9156c3a9cb&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=eea62fc4e0174ba39b800050ebe7ff5d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:07.532422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4b56a-68ad-4800-bf38-f32e6393c3e3 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:07.532950 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4b56a-68ad-4800-bf38-f32e6393c3e3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:07.532950 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4b56a-68ad-4800-bf38-f32e6393c3e3 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:07.534319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 498/983] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:07.540019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa21b1-384e-477f-86a0-de9d702bb8dc None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:07.540455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa21b1-384e-477f-86a0-de9d702bb8dc None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:07.540624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa21b1-384e-477f-86a0-de9d702bb8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:07.560814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-54fa21b1-384e-477f-86a0-de9d702bb8dc None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:07.562052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-54fa21b1-384e-477f-86a0-de9d702bb8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:07.633957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 486/984] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:07 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:07.657392 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4b56a-68ad-4800-bf38-f32e6393c3e3 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:07.684488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:07.765580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4b56a-68ad-4800-bf38-f32e6393c3e3 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:07.802817 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab381f-df41-4a8f-b7d3-ad1f976b1faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:07.803581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab381f-df41-4a8f-b7d3-ad1f976b1faf None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:07.803866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab381f-df41-4a8f-b7d3-ad1f976b1faf None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:07.804181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab381f-df41-4a8f-b7d3-ad1f976b1faf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:07.804889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab381f-df41-4a8f-b7d3-ad1f976b1faf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:07.806263 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab381f-df41-4a8f-b7d3-ad1f976b1faf None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-e9ab381f-df41-4a8f-b7d3-ad1f976b1faf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:07.818791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:07.819835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:07.820731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 499/985] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 12 03:37:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:07.826166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa21b1-384e-477f-86a0-de9d702bb8dc None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:07.927364 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa21b1-384e-477f-86a0-de9d702bb8dc None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:07.945583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:07.972451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357430d2-87b8-4d77-a542-afa631da2164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:07.973149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357430d2-87b8-4d77-a542-afa631da2164 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:07.973426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357430d2-87b8-4d77-a542-afa631da2164 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:07.973710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357430d2-87b8-4d77-a542-afa631da2164 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:07.974466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357430d2-87b8-4d77-a542-afa631da2164 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:07.974801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357430d2-87b8-4d77-a542-afa631da2164 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:07.976211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357430d2-87b8-4d77-a542-afa631da2164 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:07.985884 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 487/986] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 12 03:37:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:07.987155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab381f-df41-4a8f-b7d3-ad1f976b1faf None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:08.079005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:08.092231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab381f-df41-4a8f-b7d3-ad1f976b1faf None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:08.125853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522e053-3e5a-4c85-96ce-8e4f85041379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:08.126550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522e053-3e5a-4c85-96ce-8e4f85041379 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:08.126836 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522e053-3e5a-4c85-96ce-8e4f85041379 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:08.127110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522e053-3e5a-4c85-96ce-8e4f85041379 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:08.127842 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522e053-3e5a-4c85-96ce-8e4f85041379 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:08.128186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522e053-3e5a-4c85-96ce-8e4f85041379 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:08.129592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522e053-3e5a-4c85-96ce-8e4f85041379 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:08.135277 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 500/987] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 12 03:37:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:08.141305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357430d2-87b8-4d77-a542-afa631da2164 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:08.240230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357430d2-87b8-4d77-a542-afa631da2164 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:08.274747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a3c2f-1cb0-4138-a6e2-aa10ef8ffa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:08.275423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a3c2f-1cb0-4138-a6e2-aa10ef8ffa4c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:08.275681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a3c2f-1cb0-4138-a6e2-aa10ef8ffa4c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:08.275959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a3c2f-1cb0-4138-a6e2-aa10ef8ffa4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:08.278574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a3c2f-1cb0-4138-a6e2-aa10ef8ffa4c None admin] RBAC: Adding query filter params (interface=admin, service_id=0ebb232015f24127a1a33120769eaef2, region_id=RegionOne) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:08.278574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a3c2f-1cb0-4138-a6e2-aa10ef8ffa4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:08.278574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a3c2f-1cb0-4138-a6e2-aa10ef8ffa4c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:08.286169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 488/988] 10.222.0.20 () {56 vars in 1249 bytes} [Tue Jul 12 03:37:08 2022] GET /identity/v3/endpoints?service_id=0ebb232015f24127a1a33120769eaef2&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:08.291573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522e053-3e5a-4c85-96ce-8e4f85041379 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:08.399470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522e053-3e5a-4c85-96ce-8e4f85041379 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:08.436965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74defa48-5574-4fe5-9be5-6bb94dd4dec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:08.437638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74defa48-5574-4fe5-9be5-6bb94dd4dec7 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:08.437921 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74defa48-5574-4fe5-9be5-6bb94dd4dec7 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:08.438199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74defa48-5574-4fe5-9be5-6bb94dd4dec7 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:08.438878 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74defa48-5574-4fe5-9be5-6bb94dd4dec7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:08.440299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74defa48-5574-4fe5-9be5-6bb94dd4dec7 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:08.446914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 501/989] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 12 03:37:08 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:37:09.292535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1b73b-9f6f-4ab4-baa9-eb51e5549541 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:09.293039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1b73b-9f6f-4ab4-baa9-eb51e5549541 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:09.293039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1b73b-9f6f-4ab4-baa9-eb51e5549541 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:09.294589 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 489/990] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:09.299789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f4409-98d0-4781-8507-9590987cb406 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:09.300272 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f4409-98d0-4781-8507-9590987cb406 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:09.300345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f4409-98d0-4781-8507-9590987cb406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:09.317810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-b37f4409-98d0-4781-8507-9590987cb406 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:09.319025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b37f4409-98d0-4781-8507-9590987cb406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:09.330066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:09.351871 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65c757-1cd0-428b-92c1-9b81520ee3dc None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:09.352172 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65c757-1cd0-428b-92c1-9b81520ee3dc None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:09.352442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65c757-1cd0-428b-92c1-9b81520ee3dc None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:09.353961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 490/991] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:37:09.359389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f667962-40b9-42c4-bd9b-20200f26957b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:09.359679 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f667962-40b9-42c4-bd9b-20200f26957b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:09.359959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f667962-40b9-42c4-bd9b-20200f26957b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:09.377392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6f667962-40b9-42c4-bd9b-20200f26957b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:09.378610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6f667962-40b9-42c4-bd9b-20200f26957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:09.392579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 502/992] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:09.415401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f4409-98d0-4781-8507-9590987cb406 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:09.416522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:09.449835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 491/993] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:09.471770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f667962-40b9-42c4-bd9b-20200f26957b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:09.528900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f4409-98d0-4781-8507-9590987cb406 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:09.571236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecc070-0d11-4739-a4fe-ad51316c28bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:09.572056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecc070-0d11-4739-a4fe-ad51316c28bc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:09.572348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecc070-0d11-4739-a4fe-ad51316c28bc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:09.572650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecc070-0d11-4739-a4fe-ad51316c28bc None admin] PATH_INFO: `/v3/services/compute` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:09.573274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecc070-0d11-4739-a4fe-ad51316c28bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:09.574675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecc070-0d11-4739-a4fe-ad51316c28bc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:09.579719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f667962-40b9-42c4-bd9b-20200f26957b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-bcecc070-0d11-4739-a4fe-ad51316c28bc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:09.586800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:09.587820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:09.588876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 503/994] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:37:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:37:09.593979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecc070-0d11-4739-a4fe-ad51316c28bc None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:09.607524 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:09.620453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9aaa6a-49aa-40a7-bced-8e63bc868554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:09.621231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aaa6a-49aa-40a7-bced-8e63bc868554 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:09.621517 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aaa6a-49aa-40a7-bced-8e63bc868554 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:09.621790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aaa6a-49aa-40a7-bced-8e63bc868554 None admin] PATH_INFO: `/v3/services/image` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:09.622501 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9aaa6a-49aa-40a7-bced-8e63bc868554 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:09.623892 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9aaa6a-49aa-40a7-bced-8e63bc868554 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-fc9aaa6a-49aa-40a7-bced-8e63bc868554 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:09.635953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:09.637155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:09.638246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 492/995] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Jul 12 03:37:09 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:37:09.644159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9aaa6a-49aa-40a7-bced-8e63bc868554 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:09.682175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:09.710423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecc070-0d11-4739-a4fe-ad51316c28bc None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:09.744550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fbce8-519c-4662-b609-61a94ef2eb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:09.745208 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fbce8-519c-4662-b609-61a94ef2eb60 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:09.745495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fbce8-519c-4662-b609-61a94ef2eb60 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:09.745775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fbce8-519c-4662-b609-61a94ef2eb60 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:09.746479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fbce8-519c-4662-b609-61a94ef2eb60 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:09.746788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fbce8-519c-4662-b609-61a94ef2eb60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:09.748312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fbce8-519c-4662-b609-61a94ef2eb60 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:09.756191 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9aaa6a-49aa-40a7-bced-8e63bc868554 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:09.757936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 504/996] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:37:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:09.759256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fbce8-519c-4662-b609-61a94ef2eb60 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:09.791647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb646ac-4e99-4304-91d6-6b2f662d1973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:09.792555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb646ac-4e99-4304-91d6-6b2f662d1973 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:09.792843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb646ac-4e99-4304-91d6-6b2f662d1973 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:09.793124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb646ac-4e99-4304-91d6-6b2f662d1973 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:09.793834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb646ac-4e99-4304-91d6-6b2f662d1973 None admin] RBAC: Adding query filter params (name=image) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:09.794224 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb646ac-4e99-4304-91d6-6b2f662d1973 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:09.795611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb646ac-4e99-4304-91d6-6b2f662d1973 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:09.800717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 493/997] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:09.806198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb646ac-4e99-4304-91d6-6b2f662d1973 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:09.839466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:09.872276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fbce8-519c-4662-b609-61a94ef2eb60 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:09.902738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:09.905983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb646ac-4e99-4304-91d6-6b2f662d1973 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:09.913099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635a6a1-3c7d-42d9-a29e-a5bc96a2b9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:09.913772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635a6a1-3c7d-42d9-a29e-a5bc96a2b9af None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:09.914054 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635a6a1-3c7d-42d9-a29e-a5bc96a2b9af None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:09.914331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635a6a1-3c7d-42d9-a29e-a5bc96a2b9af None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:09.915067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635a6a1-3c7d-42d9-a29e-a5bc96a2b9af None admin] RBAC: Adding query filter params (type=compute) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:09.915381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635a6a1-3c7d-42d9-a29e-a5bc96a2b9af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:09.916817 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635a6a1-3c7d-42d9-a29e-a5bc96a2b9af None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:09.926769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 505/998] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:37:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:09.927608 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635a6a1-3c7d-42d9-a29e-a5bc96a2b9af None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:09.945176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef821bd-66a4-41c7-9b88-6b09cbf2d54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:09.945906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef821bd-66a4-41c7-9b88-6b09cbf2d54e None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:09.946190 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef821bd-66a4-41c7-9b88-6b09cbf2d54e None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:09.946460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef821bd-66a4-41c7-9b88-6b09cbf2d54e None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:09.947224 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef821bd-66a4-41c7-9b88-6b09cbf2d54e None admin] RBAC: Adding query filter params (type=image) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:09.947532 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef821bd-66a4-41c7-9b88-6b09cbf2d54e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:09.949031 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef821bd-66a4-41c7-9b88-6b09cbf2d54e None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:09.964039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 494/999] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:09 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:10.053278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635a6a1-3c7d-42d9-a29e-a5bc96a2b9af None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:10.089809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607641a3-a57b-43da-a5f6-8dbc9309f723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:10.090422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607641a3-a57b-43da-a5f6-8dbc9309f723 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:10.090710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607641a3-a57b-43da-a5f6-8dbc9309f723 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:10.091033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607641a3-a57b-43da-a5f6-8dbc9309f723 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:10.091837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607641a3-a57b-43da-a5f6-8dbc9309f723 None admin] RBAC: Adding query filter params (interface=public, service_id=076a5ae1af604a7699466491020972a2, region_id=RegionOne) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:10.092183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607641a3-a57b-43da-a5f6-8dbc9309f723 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:10.093584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607641a3-a57b-43da-a5f6-8dbc9309f723 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:10.100548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 506/1000] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 12 03:37:09 2022] GET /identity/v3/endpoints?service_id=076a5ae1af604a7699466491020972a2&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:10.105062 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef821bd-66a4-41c7-9b88-6b09cbf2d54e None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:10.210139 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef821bd-66a4-41c7-9b88-6b09cbf2d54e None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:10.245247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb381f9-b289-4726-ae95-5b501fdabf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:10.245924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb381f9-b289-4726-ae95-5b501fdabf45 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:10.246208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb381f9-b289-4726-ae95-5b501fdabf45 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:10.246477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb381f9-b289-4726-ae95-5b501fdabf45 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:10.247207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb381f9-b289-4726-ae95-5b501fdabf45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:10.248645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb381f9-b289-4726-ae95-5b501fdabf45 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:10.255244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 495/1001] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 12 03:37:10 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:37:10.468446 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a6c44-7219-4e3e-8fbd-be7f4e5695a5 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:10.468980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a6c44-7219-4e3e-8fbd-be7f4e5695a5 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:10.468980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a6c44-7219-4e3e-8fbd-be7f4e5695a5 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:10.470310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 507/1002] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:10.475556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01634d-d27b-427c-9d6e-0c6826cc0cee None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:10.475977 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01634d-d27b-427c-9d6e-0c6826cc0cee None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:10.476113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc01634d-d27b-427c-9d6e-0c6826cc0cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:10.493193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-fc01634d-d27b-427c-9d6e-0c6826cc0cee None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:10.494359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fc01634d-d27b-427c-9d6e-0c6826cc0cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:10.518950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:10.568032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 496/1003] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:10.588684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a6c44-7219-4e3e-8fbd-be7f4e5695a5 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:10.591911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:10.698139 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a6c44-7219-4e3e-8fbd-be7f4e5695a5 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:10.733752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1e119-d348-4058-b191-4aaa15fe2fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:10.734458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1e119-d348-4058-b191-4aaa15fe2fb3 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:10.734696 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1e119-d348-4058-b191-4aaa15fe2fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:10.734994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1e119-d348-4058-b191-4aaa15fe2fb3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:10.735624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1e119-d348-4058-b191-4aaa15fe2fb3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:10.737014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1e119-d348-4058-b191-4aaa15fe2fb3 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-b6d1e119-d348-4058-b191-4aaa15fe2fb3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:10.748928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:10.749906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 03:37:10.750736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:10.752125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 508/1004] 10.222.0.20 () {56 vars in 1126 bytes} [Tue Jul 12 03:37:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:10.756171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01634d-d27b-427c-9d6e-0c6826cc0cee None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:10.860752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc01634d-d27b-427c-9d6e-0c6826cc0cee None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:10.895811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68060ada-7a48-48ca-abb2-25c2fe5398d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:10.896522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68060ada-7a48-48ca-abb2-25c2fe5398d6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:10.896806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68060ada-7a48-48ca-abb2-25c2fe5398d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:10.897080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68060ada-7a48-48ca-abb2-25c2fe5398d6 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:10.897937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68060ada-7a48-48ca-abb2-25c2fe5398d6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:10.898296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68060ada-7a48-48ca-abb2-25c2fe5398d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:10.899754 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68060ada-7a48-48ca-abb2-25c2fe5398d6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:10.909433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 497/1005] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 12 03:37:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:10.910712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1e119-d348-4058-b191-4aaa15fe2fb3 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:10.986645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:11.024029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1e119-d348-4058-b191-4aaa15fe2fb3 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:11.059743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347cbd1-9774-47c6-97c5-452145fd712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:11.060453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347cbd1-9774-47c6-97c5-452145fd712a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:11.060736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347cbd1-9774-47c6-97c5-452145fd712a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:11.061008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347cbd1-9774-47c6-97c5-452145fd712a None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:11.061723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347cbd1-9774-47c6-97c5-452145fd712a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:11.062027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347cbd1-9774-47c6-97c5-452145fd712a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:11.063447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347cbd1-9774-47c6-97c5-452145fd712a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:11.069310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 509/1006] 10.222.0.20 () {56 vars in 1122 bytes} [Tue Jul 12 03:37:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:11.074843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68060ada-7a48-48ca-abb2-25c2fe5398d6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:11.099733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:11.174683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68060ada-7a48-48ca-abb2-25c2fe5398d6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:11.207477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fdd6e-391c-489c-8233-4a79a12e5c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:11.208153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fdd6e-391c-489c-8233-4a79a12e5c29 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:11.208435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fdd6e-391c-489c-8233-4a79a12e5c29 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:11.208704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fdd6e-391c-489c-8233-4a79a12e5c29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:11.209436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fdd6e-391c-489c-8233-4a79a12e5c29 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:11.210799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fdd6e-391c-489c-8233-4a79a12e5c29 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:11.226530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 498/1007] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 12 03:37:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:37:11.982496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bc1f6-2674-4f43-b878-52924c4e9c71 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:11.983422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bc1f6-2674-4f43-b878-52924c4e9c71 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:11.983709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bc1f6-2674-4f43-b878-52924c4e9c71 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:11.985244 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 510/1008] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:11.990549 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5899edf-c5c9-4496-9cde-180ebd228f42 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:11.990991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5899edf-c5c9-4496-9cde-180ebd228f42 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:11.991070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5899edf-c5c9-4496-9cde-180ebd228f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:12.008737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e5899edf-c5c9-4496-9cde-180ebd228f42 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:12.010300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e5899edf-c5c9-4496-9cde-180ebd228f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:12.084934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 499/1009] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:11 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:12.108572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bc1f6-2674-4f43-b878-52924c4e9c71 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:12.212724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bc1f6-2674-4f43-b878-52924c4e9c71 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:12.241636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:12.248857 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e73864-b65e-4a87-b5de-277d3ba51f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:12.249522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e73864-b65e-4a87-b5de-277d3ba51f2c None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:12.249810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e73864-b65e-4a87-b5de-277d3ba51f2c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:12.250086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e73864-b65e-4a87-b5de-277d3ba51f2c None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:12.250821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e73864-b65e-4a87-b5de-277d3ba51f2c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:12.252231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e73864-b65e-4a87-b5de-277d3ba51f2c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:12.268392 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 511/1010] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:37:12 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:37:12.295304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981fc34-3637-429c-b06f-440a867fd34e None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:12.295667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981fc34-3637-429c-b06f-440a867fd34e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:12.295890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981fc34-3637-429c-b06f-440a867fd34e None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:12.297608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 500/1011] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:12.303447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9c7c5-fe75-423b-a9c6-718534d78b8b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:12.303794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9c7c5-fe75-423b-a9c6-718534d78b8b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:12.303996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9c7c5-fe75-423b-a9c6-718534d78b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:12.322855 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-dae9c7c5-fe75-423b-a9c6-718534d78b8b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:12.324198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-dae9c7c5-fe75-423b-a9c6-718534d78b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:12.405874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 512/1012] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:12 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:37:12.430373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981fc34-3637-429c-b06f-440a867fd34e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:12.442058 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:12.539632 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981fc34-3637-429c-b06f-440a867fd34e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:12.573635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e0e02-2eff-4acf-9268-1475389db32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:12.574416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e0e02-2eff-4acf-9268-1475389db32c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:12.574701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e0e02-2eff-4acf-9268-1475389db32c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:12.574976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e0e02-2eff-4acf-9268-1475389db32c None admin] PATH_INFO: `/v3/services/compute` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:12.575653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e0e02-2eff-4acf-9268-1475389db32c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:12.577068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e0e02-2eff-4acf-9268-1475389db32c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-2d2e0e02-2eff-4acf-9268-1475389db32c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:12.588636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:12.589694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:12.595810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 501/1013] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:37:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:12.598676 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9c7c5-fe75-423b-a9c6-718534d78b8b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:12.720255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9c7c5-fe75-423b-a9c6-718534d78b8b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:12.738130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:12.760598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c009b50-7a07-4956-8fb1-24d6c91d5083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:12.761268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c009b50-7a07-4956-8fb1-24d6c91d5083 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:12.761551 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c009b50-7a07-4956-8fb1-24d6c91d5083 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:12.761830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c009b50-7a07-4956-8fb1-24d6c91d5083 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:12.764119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c009b50-7a07-4956-8fb1-24d6c91d5083 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:12.764119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c009b50-7a07-4956-8fb1-24d6c91d5083 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:12.764330 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c009b50-7a07-4956-8fb1-24d6c91d5083 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:12.774329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 513/1014] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:37:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:12.775724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e0e02-2eff-4acf-9268-1475389db32c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:12.849828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:12.883430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e0e02-2eff-4acf-9268-1475389db32c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:12.918365 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dcc64-6bdb-4022-b175-ee2d92b79420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:12.919186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dcc64-6bdb-4022-b175-ee2d92b79420 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:12.919475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dcc64-6bdb-4022-b175-ee2d92b79420 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:12.919754 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dcc64-6bdb-4022-b175-ee2d92b79420 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:12.920497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dcc64-6bdb-4022-b175-ee2d92b79420 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:12.920811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dcc64-6bdb-4022-b175-ee2d92b79420 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:12.922314 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dcc64-6bdb-4022-b175-ee2d92b79420 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:12.927687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 502/1015] 10.222.0.20 () {56 vars in 1112 bytes} [Tue Jul 12 03:37:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:12.933364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c009b50-7a07-4956-8fb1-24d6c91d5083 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:13.046383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c009b50-7a07-4956-8fb1-24d6c91d5083 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:13.080392 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169b6b3-a5d7-47a6-82a4-df5ce3a8361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:13.081041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169b6b3-a5d7-47a6-82a4-df5ce3a8361f None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:13.081334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169b6b3-a5d7-47a6-82a4-df5ce3a8361f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:13.081610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169b6b3-a5d7-47a6-82a4-df5ce3a8361f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:13.082352 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169b6b3-a5d7-47a6-82a4-df5ce3a8361f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:13.083791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169b6b3-a5d7-47a6-82a4-df5ce3a8361f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:13.108129 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 514/1016] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 12 03:37:12 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:37:13.249343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1e357-4b33-4c6c-aa7c-a71b4d3f98b6 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:13.249869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1e357-4b33-4c6c-aa7c-a71b4d3f98b6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:13.249869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1e357-4b33-4c6c-aa7c-a71b4d3f98b6 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:13.251970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 503/1017] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:13.256722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed03fa-4871-4cfa-9da9-0b6733c7a0e7 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:13.256989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed03fa-4871-4cfa-9da9-0b6733c7a0e7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:13.257273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed03fa-4871-4cfa-9da9-0b6733c7a0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:13.275252 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-19ed03fa-4871-4cfa-9da9-0b6733c7a0e7 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:13.276484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-19ed03fa-4871-4cfa-9da9-0b6733c7a0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:13.306818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:13.353833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 515/1018] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:13.377158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1e357-4b33-4c6c-aa7c-a71b4d3f98b6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:13.483330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1e357-4b33-4c6c-aa7c-a71b4d3f98b6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:13.516644 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5e93b-d0cc-43e4-aea6-187d7979687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:13.517454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5e93b-d0cc-43e4-aea6-187d7979687f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:13.517738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5e93b-d0cc-43e4-aea6-187d7979687f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:13.518022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5e93b-d0cc-43e4-aea6-187d7979687f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:13.519238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5e93b-d0cc-43e4-aea6-187d7979687f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:13.520716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5e93b-d0cc-43e4-aea6-187d7979687f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:13.522579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 504/1019] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:13.528350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed03fa-4871-4cfa-9da9-0b6733c7a0e7 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:13.629528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed03fa-4871-4cfa-9da9-0b6733c7a0e7 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:13.670220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e83819-b81e-4718-8484-33e2cf6ed2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:13.670889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e83819-b81e-4718-8484-33e2cf6ed2f8 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:13.671174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e83819-b81e-4718-8484-33e2cf6ed2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:13.671452 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e83819-b81e-4718-8484-33e2cf6ed2f8 None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:13.672201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e83819-b81e-4718-8484-33e2cf6ed2f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:13.673651 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e83819-b81e-4718-8484-33e2cf6ed2f8 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:13.703825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 516/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:37:13 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:37:14.313745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223787f-e212-4635-856b-4ad4f0fb6635 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:14.314288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223787f-e212-4635-856b-4ad4f0fb6635 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:14.314288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223787f-e212-4635-856b-4ad4f0fb6635 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:14.315624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 505/1021] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:37:14.321659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5411b-d85b-4cb0-b0c4-9cb7646f0fb0 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:14.322218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5411b-d85b-4cb0-b0c4-9cb7646f0fb0 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:14.322218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5411b-d85b-4cb0-b0c4-9cb7646f0fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:14.341869 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2bd5411b-d85b-4cb0-b0c4-9cb7646f0fb0 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:14.343056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2bd5411b-d85b-4cb0-b0c4-9cb7646f0fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:14.352846 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:14.412182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 517/1022] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:14.433541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223787f-e212-4635-856b-4ad4f0fb6635 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:14.437476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:14.537189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223787f-e212-4635-856b-4ad4f0fb6635 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:14.569562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27023858-29ac-4705-8210-e71c10d90e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:14.570296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27023858-29ac-4705-8210-e71c10d90e0f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:14.570577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27023858-29ac-4705-8210-e71c10d90e0f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:14.570851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27023858-29ac-4705-8210-e71c10d90e0f None admin] PATH_INFO: `/v3/services/image` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:14.571567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27023858-29ac-4705-8210-e71c10d90e0f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:14.572968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27023858-29ac-4705-8210-e71c10d90e0f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-27023858-29ac-4705-8210-e71c10d90e0f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:14.584451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:14.585641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:14.586608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 506/1023] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Jul 12 03:37:14 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:14.592084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5411b-d85b-4cb0-b0c4-9cb7646f0fb0 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:14.688129 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5411b-d85b-4cb0-b0c4-9cb7646f0fb0 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:14.722730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f0202-2d74-4baa-856b-d76a4b3c6421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:14.723477 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f0202-2d74-4baa-856b-d76a4b3c6421 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:14.723776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f0202-2d74-4baa-856b-d76a4b3c6421 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:14.724161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f0202-2d74-4baa-856b-d76a4b3c6421 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:14.725341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f0202-2d74-4baa-856b-d76a4b3c6421 None admin] RBAC: Adding query filter params (name=image) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:14.725725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f0202-2d74-4baa-856b-d76a4b3c6421 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:14.728032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f0202-2d74-4baa-856b-d76a4b3c6421 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:14.733435 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 518/1024] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:14.739415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27023858-29ac-4705-8210-e71c10d90e0f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:14.813724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:14.850463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27023858-29ac-4705-8210-e71c10d90e0f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:14.889478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ec135-6155-4231-8bba-390ed9f8c3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:14.890151 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ec135-6155-4231-8bba-390ed9f8c3c0 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:14.890438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ec135-6155-4231-8bba-390ed9f8c3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:14.890713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ec135-6155-4231-8bba-390ed9f8c3c0 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:14.891416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ec135-6155-4231-8bba-390ed9f8c3c0 None admin] RBAC: Adding query filter params (type=image) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:14.891728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ec135-6155-4231-8bba-390ed9f8c3c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:14.893157 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ec135-6155-4231-8bba-390ed9f8c3c0 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:14.898440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 507/1025] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:14.904435 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f0202-2d74-4baa-856b-d76a4b3c6421 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:14.915499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:15.007708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f0202-2d74-4baa-856b-d76a4b3c6421 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:15.043143 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033db0a1-49fb-41e2-a196-daf6ca6c98a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:15.043866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033db0a1-49fb-41e2-a196-daf6ca6c98a4 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:15.044198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033db0a1-49fb-41e2-a196-daf6ca6c98a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:15.044472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033db0a1-49fb-41e2-a196-daf6ca6c98a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:15.045260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033db0a1-49fb-41e2-a196-daf6ca6c98a4 None admin] RBAC: Adding query filter params (interface=public, service_id=d03465f946ed47618adff25909f1726f, region_id=RegionOne) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:15.045575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033db0a1-49fb-41e2-a196-daf6ca6c98a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:15.046958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033db0a1-49fb-41e2-a196-daf6ca6c98a4 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:15.052553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 519/1026] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 12 03:37:14 2022] GET /identity/v3/endpoints?service_id=d03465f946ed47618adff25909f1726f&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:15.058131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ec135-6155-4231-8bba-390ed9f8c3c0 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:15.164326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ec135-6155-4231-8bba-390ed9f8c3c0 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:15.193653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:15.199580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d927a5f-d92c-44cf-92a8-6fad90cf7c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:15.200352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d927a5f-d92c-44cf-92a8-6fad90cf7c10 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:15.200636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d927a5f-d92c-44cf-92a8-6fad90cf7c10 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:15.200910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d927a5f-d92c-44cf-92a8-6fad90cf7c10 None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:15.201578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d927a5f-d92c-44cf-92a8-6fad90cf7c10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:15.202999 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d927a5f-d92c-44cf-92a8-6fad90cf7c10 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:15.210138 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 508/1027] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 12 03:37:15 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:37:15.268193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351c129-91bf-49fd-981f-6635f68f8aa3 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:15.268543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351c129-91bf-49fd-981f-6635f68f8aa3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:15.268737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351c129-91bf-49fd-981f-6635f68f8aa3 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:15.270316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 520/1028] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:15.275979 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03327f-080d-4e36-9241-4e6dd1f6d999 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:15.276373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03327f-080d-4e36-9241-4e6dd1f6d999 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:15.276614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03327f-080d-4e36-9241-4e6dd1f6d999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:15.295947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-af03327f-080d-4e36-9241-4e6dd1f6d999 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:15.297204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-af03327f-080d-4e36-9241-4e6dd1f6d999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:15.381546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 509/1029] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:15.405458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351c129-91bf-49fd-981f-6635f68f8aa3 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:15.482680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:15.515220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351c129-91bf-49fd-981f-6635f68f8aa3 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:15.549924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3deeb-7634-457b-b0ab-8b45f1cbf4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:15.550739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3deeb-7634-457b-b0ab-8b45f1cbf4f4 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:15.551026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3deeb-7634-457b-b0ab-8b45f1cbf4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:15.551299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3deeb-7634-457b-b0ab-8b45f1cbf4f4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:15.562097 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3deeb-7634-457b-b0ab-8b45f1cbf4f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-81d3deeb-7634-457b-b0ab-8b45f1cbf4f4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:37:15.570195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:15.571128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 521/1030] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Jul 12 03:37:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:15.578747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03327f-080d-4e36-9241-4e6dd1f6d999 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:15.686918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03327f-080d-4e36-9241-4e6dd1f6d999 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:15.713959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:15.723980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017e5c9-37d6-422d-81e3-9a35614e2ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:15.724699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017e5c9-37d6-422d-81e3-9a35614e2ada None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:15.724970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017e5c9-37d6-422d-81e3-9a35614e2ada None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:15.725255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017e5c9-37d6-422d-81e3-9a35614e2ada None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:15.726020 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017e5c9-37d6-422d-81e3-9a35614e2ada None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:15.726376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017e5c9-37d6-422d-81e3-9a35614e2ada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:15.727791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017e5c9-37d6-422d-81e3-9a35614e2ada None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:15.741231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 510/1031] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 12 03:37:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:15.742554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3deeb-7634-457b-b0ab-8b45f1cbf4f4 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:15.792859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e54ee9-b210-489b-8f10-8d72b7153dc8 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:15.793183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e54ee9-b210-489b-8f10-8d72b7153dc8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:15.793449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e54ee9-b210-489b-8f10-8d72b7153dc8 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:15.795095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 511/1032] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:37:15.799817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdde25b-b621-43e7-a9af-f3a78b0b5145 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:15.800149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdde25b-b621-43e7-a9af-f3a78b0b5145 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:15.800429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdde25b-b621-43e7-a9af-f3a78b0b5145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:15.818017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1bdde25b-b621-43e7-a9af-f3a78b0b5145 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:15.819201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1bdde25b-b621-43e7-a9af-f3a78b0b5145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:15.853445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3deeb-7634-457b-b0ab-8b45f1cbf4f4 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:15.886658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:15.900044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 512/1033] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:37:15.900265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807d4f5-a319-4d1c-acfd-7f3117a10af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:15.900265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807d4f5-a319-4d1c-acfd-7f3117a10af3 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:15.900265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807d4f5-a319-4d1c-acfd-7f3117a10af3 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:15.900265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807d4f5-a319-4d1c-acfd-7f3117a10af3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:15.906641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807d4f5-a319-4d1c-acfd-7f3117a10af3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:15.908183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807d4f5-a319-4d1c-acfd-7f3117a10af3 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:15.918453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdde25b-b621-43e7-a9af-f3a78b0b5145 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-1807d4f5-a319-4d1c-acfd-7f3117a10af3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:37:15.926397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:15.927319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:15.928355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:37:15.928355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:37:15.928355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:15.928355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:15.932373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 522/1034] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:15.933839 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807d4f5-a319-4d1c-acfd-7f3117a10af3 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:16.027726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdde25b-b621-43e7-a9af-f3a78b0b5145 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:16.049747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807d4f5-a319-4d1c-acfd-7f3117a10af3 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:16.052657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:16.072692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efd4dc-3d70-4d70-ab76-2e8af7a3bf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:16.073498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efd4dc-3d70-4d70-ab76-2e8af7a3bf90 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:16.073786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efd4dc-3d70-4d70-ab76-2e8af7a3bf90 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:16.074189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efd4dc-3d70-4d70-ab76-2e8af7a3bf90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:16.075330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efd4dc-3d70-4d70-ab76-2e8af7a3bf90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:16.076758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efd4dc-3d70-4d70-ab76-2e8af7a3bf90 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:16.078530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 513/1035] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:16.083942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efd4dc-3d70-4d70-ab76-2e8af7a3bf90 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:16.084889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80865324-c6d9-47bf-ac04-38576cb995f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:16.085548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80865324-c6d9-47bf-ac04-38576cb995f3 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:16.085827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80865324-c6d9-47bf-ac04-38576cb995f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:16.086097 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80865324-c6d9-47bf-ac04-38576cb995f3 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:16.086927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80865324-c6d9-47bf-ac04-38576cb995f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:16.087230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80865324-c6d9-47bf-ac04-38576cb995f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:16.088618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80865324-c6d9-47bf-ac04-38576cb995f3 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:16.105360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 523/1036] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:16.106502 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80865324-c6d9-47bf-ac04-38576cb995f3 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:16.156214 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:16.195194 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efd4dc-3d70-4d70-ab76-2e8af7a3bf90 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:16.214449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80865324-c6d9-47bf-ac04-38576cb995f3 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:16.235766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884ae1c-b870-46f1-aaf8-b4b7997f077b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:16.236547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884ae1c-b870-46f1-aaf8-b4b7997f077b None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:16.236766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884ae1c-b870-46f1-aaf8-b4b7997f077b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:16.237019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884ae1c-b870-46f1-aaf8-b4b7997f077b None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:16.237724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884ae1c-b870-46f1-aaf8-b4b7997f077b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:16.239159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1884ae1c-b870-46f1-aaf8-b4b7997f077b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:16.251909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1488a3-66f0-4b7f-9d64-fc77dcf5a66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:16.252808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1488a3-66f0-4b7f-9d64-fc77dcf5a66e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:16.253088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1488a3-66f0-4b7f-9d64-fc77dcf5a66e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:16.253424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1488a3-66f0-4b7f-9d64-fc77dcf5a66e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:16.261699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 514/1037] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:37:16 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:37:16.262980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1488a3-66f0-4b7f-9d64-fc77dcf5a66e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:16.264498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1488a3-66f0-4b7f-9d64-fc77dcf5a66e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-1e1488a3-66f0-4b7f-9d64-fc77dcf5a66e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:37:16.280884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:16.282080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:16.283198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:16.283198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:16.283198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:37:16.283198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:37:16.283198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:37:16.283198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:37:16.283198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:37:16.283198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:37:16.283198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:37:16.283198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:16.283198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 524/1038] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 12 03:37:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:16.288249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884ae1c-b870-46f1-aaf8-b4b7997f077b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:16.341912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:16.408815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884ae1c-b870-46f1-aaf8-b4b7997f077b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:16.445742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd52eca-9083-4a57-aeb6-c0f08bc03fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:16.446437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd52eca-9083-4a57-aeb6-c0f08bc03fb1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:16.446738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd52eca-9083-4a57-aeb6-c0f08bc03fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:16.447011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd52eca-9083-4a57-aeb6-c0f08bc03fb1 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:16.447868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd52eca-9083-4a57-aeb6-c0f08bc03fb1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:16.448194 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd52eca-9083-4a57-aeb6-c0f08bc03fb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:16.449675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd52eca-9083-4a57-aeb6-c0f08bc03fb1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:16.468339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 515/1039] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 12 03:37:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:16.473923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1488a3-66f0-4b7f-9d64-fc77dcf5a66e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:16.579294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1488a3-66f0-4b7f-9d64-fc77dcf5a66e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:16.582475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:16.614539 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e554e-3494-4134-bcec-8ae75b2af13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:16.615182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e554e-3494-4134-bcec-8ae75b2af13e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:16.615462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e554e-3494-4134-bcec-8ae75b2af13e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:16.615731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e554e-3494-4134-bcec-8ae75b2af13e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:16.684710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e554e-3494-4134-bcec-8ae75b2af13e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:16.686172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e554e-3494-4134-bcec-8ae75b2af13e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:16.783929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 525/1040] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:37:16 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:37:16.790026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd52eca-9083-4a57-aeb6-c0f08bc03fb1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:16.896626 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:16.900668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd52eca-9083-4a57-aeb6-c0f08bc03fb1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:16.935100 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242f53d-dfa3-46e2-aff0-a2a8f9a6548b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:16.935894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242f53d-dfa3-46e2-aff0-a2a8f9a6548b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:16.936208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242f53d-dfa3-46e2-aff0-a2a8f9a6548b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:16.936485 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242f53d-dfa3-46e2-aff0-a2a8f9a6548b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:16.949194 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242f53d-dfa3-46e2-aff0-a2a8f9a6548b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:16.950672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242f53d-dfa3-46e2-aff0-a2a8f9a6548b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-4242f53d-dfa3-46e2-aff0-a2a8f9a6548b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:37:16.971832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:16.972895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:16.973822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:37:16.973822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:37:16.973822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:16.973822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:16.973822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 516/1041] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:16.978908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e554e-3494-4134-bcec-8ae75b2af13e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:16.982193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:17.081779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e554e-3494-4134-bcec-8ae75b2af13e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:17.116772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ed279-df49-4c2c-a59c-c6496da29dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:17.117565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed279-df49-4c2c-a59c-c6496da29dfc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:17.117943 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed279-df49-4c2c-a59c-c6496da29dfc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:17.118325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ed279-df49-4c2c-a59c-c6496da29dfc None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:17.119129 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ed279-df49-4c2c-a59c-c6496da29dfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:17.119525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ed279-df49-4c2c-a59c-c6496da29dfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:17.121012 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ed279-df49-4c2c-a59c-c6496da29dfc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:17.133856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 526/1042] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:17.139741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242f53d-dfa3-46e2-aff0-a2a8f9a6548b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:17.241994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242f53d-dfa3-46e2-aff0-a2a8f9a6548b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:17.248028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099ac7-79a3-4275-b302-f6f41e014791 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:17.248333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099ac7-79a3-4275-b302-f6f41e014791 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:17.248606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099ac7-79a3-4275-b302-f6f41e014791 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:17.249955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 527/1043] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:37:17.255013 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6d2f3-57b0-45a5-b8b2-b305a4632678 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:17.255364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6d2f3-57b0-45a5-b8b2-b305a4632678 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:17.255739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6d2f3-57b0-45a5-b8b2-b305a4632678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:17.272659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8ee6d2f3-57b0-45a5-b8b2-b305a4632678 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:17.273933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8ee6d2f3-57b0-45a5-b8b2-b305a4632678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:17.280287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49031a87-0a0f-4ec3-b788-5a41fe851676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:17.281088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49031a87-0a0f-4ec3-b788-5a41fe851676 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:17.281368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49031a87-0a0f-4ec3-b788-5a41fe851676 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:17.281641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49031a87-0a0f-4ec3-b788-5a41fe851676 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:17.288983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49031a87-0a0f-4ec3-b788-5a41fe851676 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:17.290805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49031a87-0a0f-4ec3-b788-5a41fe851676 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-49031a87-0a0f-4ec3-b788-5a41fe851676 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:37:17.304640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:17.305739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:17.306825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:37:17.306825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:37:17.306825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:37:17.306825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:37:17.306825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:37:17.306825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:37:17.306825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:37:17.306825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:17.310743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 517/1044] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:37:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:17.311566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49031a87-0a0f-4ec3-b788-5a41fe851676 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:17.327043 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:17.343476 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:17.353135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 528/1045] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:37:17.376135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6d2f3-57b0-45a5-b8b2-b305a4632678 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:17.422138 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49031a87-0a0f-4ec3-b788-5a41fe851676 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:17.459422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bd29e-0740-41cd-87ee-e7263e833638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:17.460114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bd29e-0740-41cd-87ee-e7263e833638 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:17.460399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bd29e-0740-41cd-87ee-e7263e833638 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:17.460670 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bd29e-0740-41cd-87ee-e7263e833638 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:17.461460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bd29e-0740-41cd-87ee-e7263e833638 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:17.461771 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bd29e-0740-41cd-87ee-e7263e833638 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:17.463207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bd29e-0740-41cd-87ee-e7263e833638 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:17.476137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 518/1046] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:37:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:17.482369 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bd29e-0740-41cd-87ee-e7263e833638 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:17.485033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6d2f3-57b0-45a5-b8b2-b305a4632678 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:17.523067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295e2a2-8467-463d-acf2-43216c510738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:17.523786 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295e2a2-8467-463d-acf2-43216c510738 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:17.524116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295e2a2-8467-463d-acf2-43216c510738 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:17.524391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295e2a2-8467-463d-acf2-43216c510738 None admin] PATH_INFO: `/v3/services/image` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:17.525132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295e2a2-8467-463d-acf2-43216c510738 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:17.526541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295e2a2-8467-463d-acf2-43216c510738 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-7295e2a2-8467-463d-acf2-43216c510738 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:17.539546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:17.540590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:17.541481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 529/1047] 10.222.0.20 () {56 vars in 1105 bytes} [Tue Jul 12 03:37:17 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:37:17.546679 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295e2a2-8467-463d-acf2-43216c510738 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:17.563241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:17.598775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bd29e-0740-41cd-87ee-e7263e833638 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:17.605536 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:17.639973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf42ee9-be23-42b7-8992-34843b86b133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:17.640778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf42ee9-be23-42b7-8992-34843b86b133 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:17.641166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf42ee9-be23-42b7-8992-34843b86b133 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:17.641611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf42ee9-be23-42b7-8992-34843b86b133 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:17.642234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf42ee9-be23-42b7-8992-34843b86b133 None admin] RBAC: Adding query filter params (role.id=eea62fc4e0174ba39b800050ebe7ff5d, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=7138318e53ff4b7885942d49b19eb0a5) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:17.642546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf42ee9-be23-42b7-8992-34843b86b133 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:17.644039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf42ee9-be23-42b7-8992-34843b86b133 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:17.649964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 519/1048] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:37:17 2022] GET /identity/v3/role_assignments?user.id=7138318e53ff4b7885942d49b19eb0a5&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=eea62fc4e0174ba39b800050ebe7ff5d => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:17.657349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295e2a2-8467-463d-acf2-43216c510738 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:17.697334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b98c1-ca1a-4253-8d3e-462e04cb5a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:17.698005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b98c1-ca1a-4253-8d3e-462e04cb5a9c None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:17.698287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b98c1-ca1a-4253-8d3e-462e04cb5a9c None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:17.698563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b98c1-ca1a-4253-8d3e-462e04cb5a9c None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:17.699285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b98c1-ca1a-4253-8d3e-462e04cb5a9c None admin] RBAC: Adding query filter params (name=image) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:17.699591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b98c1-ca1a-4253-8d3e-462e04cb5a9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:17.701096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b98c1-ca1a-4253-8d3e-462e04cb5a9c None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:17.711926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 530/1049] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:17.713287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf42ee9-be23-42b7-8992-34843b86b133 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:17.829860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf42ee9-be23-42b7-8992-34843b86b133 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:17.865593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1cb2f-1d85-446b-9d04-0637d3867c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:17.866497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1cb2f-1d85-446b-9d04-0637d3867c2d None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:17.866914 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1cb2f-1d85-446b-9d04-0637d3867c2d None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:17.867348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1cb2f-1d85-446b-9d04-0637d3867c2d None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:17.868225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1cb2f-1d85-446b-9d04-0637d3867c2d None admin] RBAC: Adding query filter params (type=image) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:17.868665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1cb2f-1d85-446b-9d04-0637d3867c2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:17.870203 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1cb2f-1d85-446b-9d04-0637d3867c2d None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:17.872578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:17.880088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 520/1050] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:17.882429 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b98c1-ca1a-4253-8d3e-462e04cb5a9c None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:17.990878 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b98c1-ca1a-4253-8d3e-462e04cb5a9c None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:18.014358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:18.027618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2934e-af20-4edc-897b-c68b32679a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:18.028495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2934e-af20-4edc-897b-c68b32679a42 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:18.028890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2934e-af20-4edc-897b-c68b32679a42 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:18.029285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2934e-af20-4edc-897b-c68b32679a42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:18.030123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2934e-af20-4edc-897b-c68b32679a42 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:18.031588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2934e-af20-4edc-897b-c68b32679a42 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:18.060031 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 531/1051] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 12 03:37:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:37:18.353489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f1e28-79d2-4bdb-a963-d19945dc8ba8 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:18.353901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f1e28-79d2-4bdb-a963-d19945dc8ba8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:18.354020 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f1e28-79d2-4bdb-a963-d19945dc8ba8 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:18.355546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 521/1052] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:18.360812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1af897-6f46-41b7-ad8c-29213e72dd22 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:18.361104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1af897-6f46-41b7-ad8c-29213e72dd22 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:18.361379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1af897-6f46-41b7-ad8c-29213e72dd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:18.379031 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-6e1af897-6f46-41b7-ad8c-29213e72dd22 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:18.380259 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6e1af897-6f46-41b7-ad8c-29213e72dd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:18.454940 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 532/1053] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:18.477619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f1e28-79d2-4bdb-a963-d19945dc8ba8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:18.581150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f1e28-79d2-4bdb-a963-d19945dc8ba8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:18.615103 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0167a-9a70-469c-9dab-7bb179721865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:18.615872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0167a-9a70-469c-9dab-7bb179721865 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:18.616179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0167a-9a70-469c-9dab-7bb179721865 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:18.616445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0167a-9a70-469c-9dab-7bb179721865 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:18.626908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0167a-9a70-469c-9dab-7bb179721865 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a0e0167a-9a70-469c-9dab-7bb179721865 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:37:18.634730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:18.635660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:18.636584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:18.636584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:18.636584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:18.636584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:18.636584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:37:18.636584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:37:18.636584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:37:18.636584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:37:18.636584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:37:18.636584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:37:18.636584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:37:18.636584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:18.638006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 522/1054] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:37:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:18.642143 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1af897-6f46-41b7-ad8c-29213e72dd22 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:18.749920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1af897-6f46-41b7-ad8c-29213e72dd22 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:18.753340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:18.787413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b6bda-18be-4103-9527-388f2275b4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:18.788106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b6bda-18be-4103-9527-388f2275b4fa None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:18.788391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b6bda-18be-4103-9527-388f2275b4fa None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:18.788664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b6bda-18be-4103-9527-388f2275b4fa None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:18.789405 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b6bda-18be-4103-9527-388f2275b4fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:18.789714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b6bda-18be-4103-9527-388f2275b4fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:18.791088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b6bda-18be-4103-9527-388f2275b4fa None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:18.800173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 533/1055] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:37:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:18.805785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0167a-9a70-469c-9dab-7bb179721865 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:18.865326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:18.908644 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0167a-9a70-469c-9dab-7bb179721865 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:18.941412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d56281-6a95-4da5-a896-932a663c8117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:18.942262 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d56281-6a95-4da5-a896-932a663c8117 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:18.942543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d56281-6a95-4da5-a896-932a663c8117 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:18.942821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d56281-6a95-4da5-a896-932a663c8117 None admin] PATH_INFO: `/v3/users/62cb6a533cb1455d8804d55acfcbc469` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:18.959960 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d56281-6a95-4da5-a896-932a663c8117 None admin] RBAC: Authorizing `identity:get_user(user_id=62cb6a533cb1455d8804d55acfcbc469)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:18.961485 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d56281-6a95-4da5-a896-932a663c8117 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:18.964156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 523/1056] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:37:18 2022] GET /identity/v3/users/62cb6a533cb1455d8804d55acfcbc469 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:18.968906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b6bda-18be-4103-9527-388f2275b4fa None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:19.071330 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b6bda-18be-4103-9527-388f2275b4fa None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:19.105951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc75eee-ab3f-47ab-8051-390215291839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:19.106670 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc75eee-ab3f-47ab-8051-390215291839 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:19.106951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc75eee-ab3f-47ab-8051-390215291839 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:19.107225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc75eee-ab3f-47ab-8051-390215291839 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:19.142168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:19.179460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc75eee-ab3f-47ab-8051-390215291839 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:19.180985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc75eee-ab3f-47ab-8051-390215291839 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:19.282522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 534/1057] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:37:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:37:19.288566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d56281-6a95-4da5-a896-932a663c8117 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:19.394068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d56281-6a95-4da5-a896-932a663c8117 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:19.411480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:19.429548 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60322b8-c58d-4886-9e7a-853d7f7916c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:19.430408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60322b8-c58d-4886-9e7a-853d7f7916c4 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:19.430730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60322b8-c58d-4886-9e7a-853d7f7916c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:19.431007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60322b8-c58d-4886-9e7a-853d7f7916c4 None admin] PATH_INFO: `/v3/projects/1e194449828e40c0aca9d578b203b4fc` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:19.432274 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60322b8-c58d-4886-9e7a-853d7f7916c4 None admin] RBAC: Authorizing `identity:get_project(project_id=1e194449828e40c0aca9d578b203b4fc)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:19.433717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60322b8-c58d-4886-9e7a-853d7f7916c4 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:19.435726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 524/1058] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:37:19 2022] GET /identity/v3/projects/1e194449828e40c0aca9d578b203b4fc => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:19.441726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc75eee-ab3f-47ab-8051-390215291839 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:19.507582 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:19.543550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc75eee-ab3f-47ab-8051-390215291839 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:19.577879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d19ed-ed49-488b-8b9e-40d74baf9f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:19.578533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d19ed-ed49-488b-8b9e-40d74baf9f9b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:19.578776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d19ed-ed49-488b-8b9e-40d74baf9f9b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:19.579085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d19ed-ed49-488b-8b9e-40d74baf9f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:19.579911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d19ed-ed49-488b-8b9e-40d74baf9f9b None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=1e194449828e40c0aca9d578b203b4fc, user.id=62cb6a533cb1455d8804d55acfcbc469) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:19.580250 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d19ed-ed49-488b-8b9e-40d74baf9f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:19.581685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d19ed-ed49-488b-8b9e-40d74baf9f9b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:19.587754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 535/1059] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:37:19 2022] GET /identity/v3/role_assignments?user.id=62cb6a533cb1455d8804d55acfcbc469&scope.project.id=1e194449828e40c0aca9d578b203b4fc&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:19.663603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85f5a8-6673-4abc-a142-b472a9116c2b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:19.664161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85f5a8-6673-4abc-a142-b472a9116c2b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:19.664161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85f5a8-6673-4abc-a142-b472a9116c2b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:19.665664 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 525/1060] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:19.671565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01456e-86f3-4c4f-833a-abe463205b4a None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:19.672282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01456e-86f3-4c4f-833a-abe463205b4a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:19.672282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01456e-86f3-4c4f-833a-abe463205b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:19.689602 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2b01456e-86f3-4c4f-833a-abe463205b4a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:19.690796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2b01456e-86f3-4c4f-833a-abe463205b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:19.769640 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 536/1061] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:19.791820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85f5a8-6673-4abc-a142-b472a9116c2b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:19.894782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85f5a8-6673-4abc-a142-b472a9116c2b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:19.934410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137a851-3eba-4ec2-aff1-9178ee912c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:19.935154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137a851-3eba-4ec2-aff1-9178ee912c45 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:19.935436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137a851-3eba-4ec2-aff1-9178ee912c45 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:19.935707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137a851-3eba-4ec2-aff1-9178ee912c45 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:19.949719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137a851-3eba-4ec2-aff1-9178ee912c45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-9137a851-3eba-4ec2-aff1-9178ee912c45 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:37:19.959436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:19.960444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:19.961308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:19.961308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:19.961308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:19.961308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:19.961308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:37:19.961308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:37:19.961308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:37:19.961308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:37:19.961308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:37:19.961308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:37:19.961308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 03:37:19.961308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:19.964395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 526/1062] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Jul 12 03:37:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:19.965943 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01456e-86f3-4c4f-833a-abe463205b4a None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:20.036604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:20.081514 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc90f6-f11d-45f8-b8ee-231652a56a60 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:20.081867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc90f6-f11d-45f8-b8ee-231652a56a60 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:20.082074 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc90f6-f11d-45f8-b8ee-231652a56a60 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:20.082338 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01456e-86f3-4c4f-833a-abe463205b4a None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:20.083423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 527/1063] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:37:20.088584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b140f-e652-48ab-a13e-9eeee62fe4ee None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:20.088948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b140f-e652-48ab-a13e-9eeee62fe4ee None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:20.089296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b140f-e652-48ab-a13e-9eeee62fe4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:20.106300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-bc8b140f-e652-48ab-a13e-9eeee62fe4ee None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:20.107509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bc8b140f-e652-48ab-a13e-9eeee62fe4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:20.122749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717aa53-26ab-446e-90ed-1ae925fa92bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:20.123410 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717aa53-26ab-446e-90ed-1ae925fa92bc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:20.123694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717aa53-26ab-446e-90ed-1ae925fa92bc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:20.123968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717aa53-26ab-446e-90ed-1ae925fa92bc None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:20.124754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717aa53-26ab-446e-90ed-1ae925fa92bc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:20.125062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717aa53-26ab-446e-90ed-1ae925fa92bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:20.126442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717aa53-26ab-446e-90ed-1ae925fa92bc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:20.140082 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 537/1064] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 12 03:37:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:20.142150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717aa53-26ab-446e-90ed-1ae925fa92bc None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:20.158063 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:20.182904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 528/1065] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:37:20.204576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b140f-e652-48ab-a13e-9eeee62fe4ee None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:20.258895 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717aa53-26ab-446e-90ed-1ae925fa92bc None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:20.286113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:20.298990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb92eb9-c43c-4de8-9f2d-eb08068dbeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:20.299784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb92eb9-c43c-4de8-9f2d-eb08068dbeda None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:20.300119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb92eb9-c43c-4de8-9f2d-eb08068dbeda None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:20.300384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb92eb9-c43c-4de8-9f2d-eb08068dbeda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:20.305338 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b140f-e652-48ab-a13e-9eeee62fe4ee None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:20.311160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb92eb9-c43c-4de8-9f2d-eb08068dbeda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:20.312667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb92eb9-c43c-4de8-9f2d-eb08068dbeda None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-bfb92eb9-c43c-4de8-9f2d-eb08068dbeda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:37:20.330868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:20.331794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:20.332709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:37:20.332709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:37:20.332709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:20.332709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:20.332709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 538/1066] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:20.337898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb92eb9-c43c-4de8-9f2d-eb08068dbeda None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:20.342822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec75368-74c8-4753-8d5f-678166f149ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:20.343481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec75368-74c8-4753-8d5f-678166f149ac None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:20.343752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec75368-74c8-4753-8d5f-678166f149ac None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:20.344036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec75368-74c8-4753-8d5f-678166f149ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:20.390592 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:20.416928 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec75368-74c8-4753-8d5f-678166f149ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:20.418411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec75368-74c8-4753-8d5f-678166f149ac None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:20.446360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb92eb9-c43c-4de8-9f2d-eb08068dbeda None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:20.479610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97327d10-64dd-4c9d-be18-3b56a884446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:20.480234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97327d10-64dd-4c9d-be18-3b56a884446e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:20.480523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97327d10-64dd-4c9d-be18-3b56a884446e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:20.480804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97327d10-64dd-4c9d-be18-3b56a884446e None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:20.481622 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97327d10-64dd-4c9d-be18-3b56a884446e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:20.481931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97327d10-64dd-4c9d-be18-3b56a884446e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:20.483311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97327d10-64dd-4c9d-be18-3b56a884446e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:20.495319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 539/1067] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:20.501075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97327d10-64dd-4c9d-be18-3b56a884446e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:20.522289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 529/1068] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:37:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 03:37:20.528992 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec75368-74c8-4753-8d5f-678166f149ac None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:20.534747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:20.604149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97327d10-64dd-4c9d-be18-3b56a884446e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:20.607026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:20.630709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec75368-74c8-4753-8d5f-678166f149ac None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:20.638615 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdc461-dea4-407e-ba40-ea9c226d3f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:20.639416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdc461-dea4-407e-ba40-ea9c226d3f7e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:20.639725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdc461-dea4-407e-ba40-ea9c226d3f7e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:20.639972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdc461-dea4-407e-ba40-ea9c226d3f7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:20.647641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdc461-dea4-407e-ba40-ea9c226d3f7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:20.649166 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdc461-dea4-407e-ba40-ea9c226d3f7e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-26fdc461-dea4-407e-ba40-ea9c226d3f7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:37:20.663610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:20.664592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:20.666256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:20.666256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:20.666256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:37:20.666256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:37:20.666256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:37:20.666256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:37:20.666256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:37:20.666256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:37:20.666256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:37:20.666256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:20.666256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 540/1069] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 12 03:37:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:20.668492 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31947df-cdc2-449f-a7fa-b8c38dcdf207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:20.669561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31947df-cdc2-449f-a7fa-b8c38dcdf207 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:20.669561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31947df-cdc2-449f-a7fa-b8c38dcdf207 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:20.669834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31947df-cdc2-449f-a7fa-b8c38dcdf207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:20.670384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdc461-dea4-407e-ba40-ea9c226d3f7e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:20.681801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31947df-cdc2-449f-a7fa-b8c38dcdf207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:20.683256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31947df-cdc2-449f-a7fa-b8c38dcdf207 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-f31947df-cdc2-449f-a7fa-b8c38dcdf207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:37:20.702430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:20.703459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:20.705006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:37:20.705006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:37:20.705006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:20.705006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:20.705006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 530/1070] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:37:20.709783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31947df-cdc2-449f-a7fa-b8c38dcdf207 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:20.771128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:20.778885 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdc461-dea4-407e-ba40-ea9c226d3f7e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:20.813920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b9179-f21a-4aa5-b87b-f04ddc8b3f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:20.814606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b9179-f21a-4aa5-b87b-f04ddc8b3f6a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:20.814876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b9179-f21a-4aa5-b87b-f04ddc8b3f6a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:20.815000 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31947df-cdc2-449f-a7fa-b8c38dcdf207 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:20.815158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b9179-f21a-4aa5-b87b-f04ddc8b3f6a None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:20.815979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b9179-f21a-4aa5-b87b-f04ddc8b3f6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:20.816336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b9179-f21a-4aa5-b87b-f04ddc8b3f6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:20.817767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b9179-f21a-4aa5-b87b-f04ddc8b3f6a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:20.835790 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 541/1071] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 12 03:37:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:20.841535 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b9179-f21a-4aa5-b87b-f04ddc8b3f6a None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:20.842248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:20.850195 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154d745-647a-43e9-a11d-d36c44e5fbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:20.850863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154d745-647a-43e9-a11d-d36c44e5fbc3 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:20.851141 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154d745-647a-43e9-a11d-d36c44e5fbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:20.851661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154d745-647a-43e9-a11d-d36c44e5fbc3 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:20.852236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154d745-647a-43e9-a11d-d36c44e5fbc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:20.852543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154d745-647a-43e9-a11d-d36c44e5fbc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:20.853914 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154d745-647a-43e9-a11d-d36c44e5fbc3 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:20.876039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 531/1072] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:20.973824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b9179-f21a-4aa5-b87b-f04ddc8b3f6a None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:21.012680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068211a-a088-4f18-87e8-c17e4be7c428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:21.013527 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068211a-a088-4f18-87e8-c17e4be7c428 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:21.013811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068211a-a088-4f18-87e8-c17e4be7c428 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:21.014082 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068211a-a088-4f18-87e8-c17e4be7c428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:21.025352 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068211a-a088-4f18-87e8-c17e4be7c428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:21.026851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068211a-a088-4f18-87e8-c17e4be7c428 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-7068211a-a088-4f18-87e8-c17e4be7c428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:37:21.044618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:21.045555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:21.046474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:37:21.046474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:37:21.046474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:21.046474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:21.046474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 542/1073] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:37:21.051594 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154d745-647a-43e9-a11d-d36c44e5fbc3 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:21.159628 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154d745-647a-43e9-a11d-d36c44e5fbc3 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:21.192947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c19f2-cd14-49c3-9e43-a8640dbf6499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:21.193620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c19f2-cd14-49c3-9e43-a8640dbf6499 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:21.193901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c19f2-cd14-49c3-9e43-a8640dbf6499 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:21.194172 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c19f2-cd14-49c3-9e43-a8640dbf6499 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:21.194892 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c19f2-cd14-49c3-9e43-a8640dbf6499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:21.195199 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c19f2-cd14-49c3-9e43-a8640dbf6499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:21.196619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c19f2-cd14-49c3-9e43-a8640dbf6499 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:21.208380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 532/1074] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:21.214081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068211a-a088-4f18-87e8-c17e4be7c428 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:21.218032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:21.316518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068211a-a088-4f18-87e8-c17e4be7c428 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:21.351307 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df1b90-33d3-4e49-bfd4-0a6355ed9ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:21.352128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df1b90-33d3-4e49-bfd4-0a6355ed9ccd None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:21.352412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df1b90-33d3-4e49-bfd4-0a6355ed9ccd None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:21.352688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df1b90-33d3-4e49-bfd4-0a6355ed9ccd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:21.359387 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df1b90-33d3-4e49-bfd4-0a6355ed9ccd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:21.360873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df1b90-33d3-4e49-bfd4-0a6355ed9ccd None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-74df1b90-33d3-4e49-bfd4-0a6355ed9ccd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:37:21.374903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:21.375810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:21.377087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:37:21.377087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:37:21.377087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:37:21.377087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:37:21.377087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:37:21.377087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:37:21.377087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:37:21.377087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:21.377087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 543/1075] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:37:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:37:21.382064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c19f2-cd14-49c3-9e43-a8640dbf6499 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:21.432681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:21.487689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c19f2-cd14-49c3-9e43-a8640dbf6499 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:21.524228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af921e-5d45-4d13-8885-1395d13b797c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:21.524881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af921e-5d45-4d13-8885-1395d13b797c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:21.525161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af921e-5d45-4d13-8885-1395d13b797c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:21.525429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af921e-5d45-4d13-8885-1395d13b797c None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:21.526205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af921e-5d45-4d13-8885-1395d13b797c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:21.526509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af921e-5d45-4d13-8885-1395d13b797c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:21.528032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af921e-5d45-4d13-8885-1395d13b797c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:21.540054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 533/1076] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:37:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:21.546370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df1b90-33d3-4e49-bfd4-0a6355ed9ccd None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:21.611713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:21.629288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b662c-f41d-42fa-b135-b6d153cf280a None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:21.629582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b662c-f41d-42fa-b135-b6d153cf280a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:21.629878 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b662c-f41d-42fa-b135-b6d153cf280a None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:21.631251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 534/1077] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:21.636900 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4121f2-5339-4b50-b6ec-98302304ceb9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:21.637186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4121f2-5339-4b50-b6ec-98302304ceb9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:21.637463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4121f2-5339-4b50-b6ec-98302304ceb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:21.651588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df1b90-33d3-4e49-bfd4-0a6355ed9ccd None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:21.654825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5f4121f2-5339-4b50-b6ec-98302304ceb9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:21.656051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5f4121f2-5339-4b50-b6ec-98302304ceb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:21.686476 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ac843-a660-47df-b137-b696a4c58f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:21.687391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ac843-a660-47df-b137-b696a4c58f87 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:21.687672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ac843-a660-47df-b137-b696a4c58f87 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:21.687941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ac843-a660-47df-b137-b696a4c58f87 None admin] PATH_INFO: `/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/7138318e53ff4b7885942d49b19eb0a5/roles/eea62fc4e0174ba39b800050ebe7ff5d` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:21.690232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ac843-a660-47df-b137-b696a4c58f87 None admin] RBAC: Authorizing `identity:create_grant(project_id=48b17053daa84e4fa94840777cb1bee6, user_id=7138318e53ff4b7885942d49b19eb0a5, role_id=eea62fc4e0174ba39b800050ebe7ff5d)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:21.691716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ac843-a660-47df-b137-b696a4c58f87 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:21.705660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 544/1078] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:37:21 2022] PUT /identity/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/7138318e53ff4b7885942d49b19eb0a5/roles/eea62fc4e0174ba39b800050ebe7ff5d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:37:21.729325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 535/1079] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:21.753313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ac843-a660-47df-b137-b696a4c58f87 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:21.875980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ac843-a660-47df-b137-b696a4c58f87 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:21.900482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:21.913362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb5a0d-c1bd-4852-9f99-08791df9d04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:21.914161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb5a0d-c1bd-4852-9f99-08791df9d04e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:21.914439 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb5a0d-c1bd-4852-9f99-08791df9d04e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:21.914713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb5a0d-c1bd-4852-9f99-08791df9d04e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:21.924745 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb5a0d-c1bd-4852-9f99-08791df9d04e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-7abb5a0d-c1bd-4852-9f99-08791df9d04e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:37:21.932184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:21.933309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:21.934267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 545/1080] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:37:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:21.940030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4121f2-5339-4b50-b6ec-98302304ceb9 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:21.984923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:22.049748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4121f2-5339-4b50-b6ec-98302304ceb9 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:22.085834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc7c9e-440b-4942-8047-d79dda98cb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:22.086532 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc7c9e-440b-4942-8047-d79dda98cb32 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:22.086819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc7c9e-440b-4942-8047-d79dda98cb32 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:22.087099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc7c9e-440b-4942-8047-d79dda98cb32 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:22.087852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc7c9e-440b-4942-8047-d79dda98cb32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:22.088201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc7c9e-440b-4942-8047-d79dda98cb32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:22.089630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc7c9e-440b-4942-8047-d79dda98cb32 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:22.098506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 536/1081] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:37:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:22.104630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb5a0d-c1bd-4852-9f99-08791df9d04e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:22.208121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb5a0d-c1bd-4852-9f99-08791df9d04e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:22.242946 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851567ab-584e-461f-a387-9f4c241c028b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:22.243822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851567ab-584e-461f-a387-9f4c241c028b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:22.244144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851567ab-584e-461f-a387-9f4c241c028b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:22.244421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851567ab-584e-461f-a387-9f4c241c028b None admin] PATH_INFO: `/v3/users/62cb6a533cb1455d8804d55acfcbc469` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:22.245658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851567ab-584e-461f-a387-9f4c241c028b None admin] RBAC: Authorizing `identity:get_user(user_id=62cb6a533cb1455d8804d55acfcbc469)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:22.247199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851567ab-584e-461f-a387-9f4c241c028b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:22.249225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 546/1082] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:37:22 2022] GET /identity/v3/users/62cb6a533cb1455d8804d55acfcbc469 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:22.255480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc7c9e-440b-4942-8047-d79dda98cb32 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:22.352908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:22.356083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc7c9e-440b-4942-8047-d79dda98cb32 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:22.391723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb07dbd-7eb8-4781-9d5d-141c41b8bfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:22.392683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb07dbd-7eb8-4781-9d5d-141c41b8bfde None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:22.392985 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb07dbd-7eb8-4781-9d5d-141c41b8bfde None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:22.393256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb07dbd-7eb8-4781-9d5d-141c41b8bfde None admin] PATH_INFO: `/v3/projects/1e194449828e40c0aca9d578b203b4fc` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:22.394429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb07dbd-7eb8-4781-9d5d-141c41b8bfde None admin] RBAC: Authorizing `identity:get_project(project_id=1e194449828e40c0aca9d578b203b4fc)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:22.395891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb07dbd-7eb8-4781-9d5d-141c41b8bfde None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:22.397653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 537/1083] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:37:22 2022] GET /identity/v3/projects/1e194449828e40c0aca9d578b203b4fc => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:22.403994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851567ab-584e-461f-a387-9f4c241c028b None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:22.458024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:22.506911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851567ab-584e-461f-a387-9f4c241c028b None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:22.541189 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02d63a-b128-4849-b455-4d39f153c536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:22.542089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02d63a-b128-4849-b455-4d39f153c536 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:22.542368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02d63a-b128-4849-b455-4d39f153c536 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:22.542628 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02d63a-b128-4849-b455-4d39f153c536 None admin] PATH_INFO: `/v3/projects/1e194449828e40c0aca9d578b203b4fc/users/62cb6a533cb1455d8804d55acfcbc469/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:22.544695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02d63a-b128-4849-b455-4d39f153c536 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e194449828e40c0aca9d578b203b4fc, user_id=62cb6a533cb1455d8804d55acfcbc469, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:22.546131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02d63a-b128-4849-b455-4d39f153c536 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:22.559246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 547/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:37:22 2022] PUT /identity/v3/projects/1e194449828e40c0aca9d578b203b4fc/users/62cb6a533cb1455d8804d55acfcbc469/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:37:23.677359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da965638-1933-4b8b-942c-7f80342bb2e0 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:23.678004 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da965638-1933-4b8b-942c-7f80342bb2e0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:23.678004 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da965638-1933-4b8b-942c-7f80342bb2e0 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:23.679242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 538/1085] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:23.684720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a3d42-110f-4e2e-a77e-73af0b56f319 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:23.685118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a3d42-110f-4e2e-a77e-73af0b56f319 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:23.685314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a3d42-110f-4e2e-a77e-73af0b56f319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:23.703138 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-c12a3d42-110f-4e2e-a77e-73af0b56f319 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:23.704381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c12a3d42-110f-4e2e-a77e-73af0b56f319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:23.772984 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 548/1086] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:23.796460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da965638-1933-4b8b-942c-7f80342bb2e0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:23.899926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da965638-1933-4b8b-942c-7f80342bb2e0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:23.934738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949ea77-bc84-4a63-8e8f-0284c4db55fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:23.935479 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949ea77-bc84-4a63-8e8f-0284c4db55fb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:23.935786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949ea77-bc84-4a63-8e8f-0284c4db55fb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:23.936165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949ea77-bc84-4a63-8e8f-0284c4db55fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:23.947304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949ea77-bc84-4a63-8e8f-0284c4db55fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-5949ea77-bc84-4a63-8e8f-0284c4db55fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:37:23.955358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:23.956440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:23.957700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 539/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Tue Jul 12 03:37:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:23.963643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a3d42-110f-4e2e-a77e-73af0b56f319 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:24.011399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:24.069628 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a3d42-110f-4e2e-a77e-73af0b56f319 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:24.103365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052f5bf-2f45-4871-b664-7d74cc454d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:24.104038 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052f5bf-2f45-4871-b664-7d74cc454d8d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:24.104345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052f5bf-2f45-4871-b664-7d74cc454d8d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:24.104604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052f5bf-2f45-4871-b664-7d74cc454d8d None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:24.105355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052f5bf-2f45-4871-b664-7d74cc454d8d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:24.105665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052f5bf-2f45-4871-b664-7d74cc454d8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:24.107179 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052f5bf-2f45-4871-b664-7d74cc454d8d None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:24.120045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 549/1088] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 12 03:37:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:24.125913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949ea77-bc84-4a63-8e8f-0284c4db55fb None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:24.157351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:24.231322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949ea77-bc84-4a63-8e8f-0284c4db55fb None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:24.266806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f3b24-1819-4bc8-9b72-29b388445e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:24.267591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f3b24-1819-4bc8-9b72-29b388445e75 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:24.267874 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f3b24-1819-4bc8-9b72-29b388445e75 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:24.268177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f3b24-1819-4bc8-9b72-29b388445e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:24.278558 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f3b24-1819-4bc8-9b72-29b388445e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:24.280034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f3b24-1819-4bc8-9b72-29b388445e75 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-ee2f3b24-1819-4bc8-9b72-29b388445e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:37:24.297489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:24.298511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:24.299561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:37:24.299561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:37:24.299561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:24.299561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:24.303318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 540/1089] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:24.304380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052f5bf-2f45-4871-b664-7d74cc454d8d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:24.410394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052f5bf-2f45-4871-b664-7d74cc454d8d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:24.412902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:24.448463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a53e47-b524-4fe8-9bc6-b96ab5b703ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:24.449129 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a53e47-b524-4fe8-9bc6-b96ab5b703ce None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:24.449409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a53e47-b524-4fe8-9bc6-b96ab5b703ce None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:24.449716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a53e47-b524-4fe8-9bc6-b96ab5b703ce None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:24.450439 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a53e47-b524-4fe8-9bc6-b96ab5b703ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:24.450740 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a53e47-b524-4fe8-9bc6-b96ab5b703ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:24.452161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a53e47-b524-4fe8-9bc6-b96ab5b703ce None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:24.473237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 550/1090] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:24.474527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f3b24-1819-4bc8-9b72-29b388445e75 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:24.558326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15f3a6-60c4-4c76-ad0e-4eebf1e5493b None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:24.558603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15f3a6-60c4-4c76-ad0e-4eebf1e5493b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:24.558902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15f3a6-60c4-4c76-ad0e-4eebf1e5493b None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:24.560412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 551/1091] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:24.563935 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:24.565378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ea77d-44e0-456d-a277-be4035a3a672 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:24.565751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ea77d-44e0-456d-a277-be4035a3a672 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:24.565921 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ea77d-44e0-456d-a277-be4035a3a672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:24.582854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-302ea77d-44e0-456d-a277-be4035a3a672 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:24.584071 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-302ea77d-44e0-456d-a277-be4035a3a672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:24.585581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f3b24-1819-4bc8-9b72-29b388445e75 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:24.619729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8ae81-4568-405e-abef-ad76db455171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:24.620550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8ae81-4568-405e-abef-ad76db455171 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:24.620835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8ae81-4568-405e-abef-ad76db455171 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:24.621107 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8ae81-4568-405e-abef-ad76db455171 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:24.629618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea8ae81-4568-405e-abef-ad76db455171 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:24.631080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea8ae81-4568-405e-abef-ad76db455171 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6ea8ae81-4568-405e-abef-ad76db455171 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:37:24.648481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:24.649441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:24.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:24.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:24.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:37:24.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:37:24.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:37:24.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:37:24.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:37:24.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:37:24.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 03:37:24.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:24.650411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 541/1092] 10.222.0.20 () {56 vars in 1093 bytes} [Tue Jul 12 03:37:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:24.651801 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 552/1093] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:37:24.655472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8ae81-4568-405e-abef-ad76db455171 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:24.674535 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ea77d-44e0-456d-a277-be4035a3a672 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:24.752386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8ae81-4568-405e-abef-ad76db455171 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:24.766421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:24.773115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ea77d-44e0-456d-a277-be4035a3a672 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:24.787116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a8753-fcb0-4c7d-a3b1-b826161978f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:24.787790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8753-fcb0-4c7d-a3b1-b826161978f8 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:24.788108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8753-fcb0-4c7d-a3b1-b826161978f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:24.788390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a8753-fcb0-4c7d-a3b1-b826161978f8 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:24.789239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a8753-fcb0-4c7d-a3b1-b826161978f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:24.789555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a8753-fcb0-4c7d-a3b1-b826161978f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:24.791023 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a8753-fcb0-4c7d-a3b1-b826161978f8 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:24.805793 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:24.807419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a021399-9a6b-4c3e-93c6-920c31afab21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:24.809040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a021399-9a6b-4c3e-93c6-920c31afab21 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:24.809040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a021399-9a6b-4c3e-93c6-920c31afab21 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:24.809321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a021399-9a6b-4c3e-93c6-920c31afab21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:24.816636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 542/1094] 10.222.0.20 () {56 vars in 1133 bytes} [Tue Jul 12 03:37:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:24.821532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a021399-9a6b-4c3e-93c6-920c31afab21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:37:24.822389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a8753-fcb0-4c7d-a3b1-b826161978f8 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-3a021399-9a6b-4c3e-93c6-920c31afab21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:37:24.829186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:24.830290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:24.831141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 553/1095] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:37:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:37:24.836668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a021399-9a6b-4c3e-93c6-920c31afab21 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:24.924755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a8753-fcb0-4c7d-a3b1-b826161978f8 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:24.938192 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a021399-9a6b-4c3e-93c6-920c31afab21 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:24.963652 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e9ba4-8e3f-43c0-b183-6bae0e020e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:24.964327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e9ba4-8e3f-43c0-b183-6bae0e020e52 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:24.964612 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e9ba4-8e3f-43c0-b183-6bae0e020e52 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:24.964888 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e9ba4-8e3f-43c0-b183-6bae0e020e52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:24.980640 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bcc62-e74e-490d-8739-d163bf754320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:24.981474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bcc62-e74e-490d-8739-d163bf754320 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:24.981757 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bcc62-e74e-490d-8739-d163bf754320 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:24.982022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bcc62-e74e-490d-8739-d163bf754320 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:24.982763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bcc62-e74e-490d-8739-d163bf754320 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:24.983064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bcc62-e74e-490d-8739-d163bf754320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:24.984523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bcc62-e74e-490d-8739-d163bf754320 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:24.992863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 554/1096] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:37:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:24.998559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bcc62-e74e-490d-8739-d163bf754320 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:25.016882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:25.029750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:25.033476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e9ba4-8e3f-43c0-b183-6bae0e020e52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:25.035299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6e9ba4-8e3f-43c0-b183-6bae0e020e52 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:25.104292 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bcc62-e74e-490d-8739-d163bf754320 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:25.135403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 543/1097] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:37:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 03:37:25.137731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66def9b5-5964-42c9-911c-ea83f585be5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:25.138613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66def9b5-5964-42c9-911c-ea83f585be5f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:25.138897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66def9b5-5964-42c9-911c-ea83f585be5f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:25.139170 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66def9b5-5964-42c9-911c-ea83f585be5f None admin] PATH_INFO: `/v3/users/62cb6a533cb1455d8804d55acfcbc469` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:25.140395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66def9b5-5964-42c9-911c-ea83f585be5f None admin] RBAC: Authorizing `identity:get_user(user_id=62cb6a533cb1455d8804d55acfcbc469)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:25.141087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e9ba4-8e3f-43c0-b183-6bae0e020e52 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:25.141842 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66def9b5-5964-42c9-911c-ea83f585be5f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:25.148248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 555/1098] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:37:24 2022] GET /identity/v3/users/62cb6a533cb1455d8804d55acfcbc469 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:25.149464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66def9b5-5964-42c9-911c-ea83f585be5f None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:25.235468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:25.239388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e9ba4-8e3f-43c0-b183-6bae0e020e52 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:25.264857 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:25.274478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6c013-e2b6-4a44-87e2-ad7f7aca8e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:25.275274 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6c013-e2b6-4a44-87e2-ad7f7aca8e9c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:25.275551 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6c013-e2b6-4a44-87e2-ad7f7aca8e9c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:25.275823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6c013-e2b6-4a44-87e2-ad7f7aca8e9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:25.286305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6c013-e2b6-4a44-87e2-ad7f7aca8e9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:25.287458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66def9b5-5964-42c9-911c-ea83f585be5f None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:25.287751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6c013-e2b6-4a44-87e2-ad7f7aca8e9c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-baa6c013-e2b6-4a44-87e2-ad7f7aca8e9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:37:25.305867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:25.306808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:25.307961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:37:25.307961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:37:25.307961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:37:25.307961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:25.307961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 544/1099] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:37:25.312899 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6c013-e2b6-4a44-87e2-ad7f7aca8e9c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:25.327617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567688b-3e09-41ae-9314-e0ae72f9a1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:25.328296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567688b-3e09-41ae-9314-e0ae72f9a1e1 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:25.328597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567688b-3e09-41ae-9314-e0ae72f9a1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:25.328880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567688b-3e09-41ae-9314-e0ae72f9a1e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:25.395870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567688b-3e09-41ae-9314-e0ae72f9a1e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:25.397379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567688b-3e09-41ae-9314-e0ae72f9a1e1 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:25.410672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6c013-e2b6-4a44-87e2-ad7f7aca8e9c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:25.444148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3fb89-2100-4c54-84f1-3c10ff36a11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:25.444831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3fb89-2100-4c54-84f1-3c10ff36a11a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:25.445109 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3fb89-2100-4c54-84f1-3c10ff36a11a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:25.445381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3fb89-2100-4c54-84f1-3c10ff36a11a None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:25.446091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3fb89-2100-4c54-84f1-3c10ff36a11a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:25.446392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3fb89-2100-4c54-84f1-3c10ff36a11a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:25.447734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3fb89-2100-4c54-84f1-3c10ff36a11a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:25.459030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 545/1100] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:25.464336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3fb89-2100-4c54-84f1-3c10ff36a11a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:25.476601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:25.482145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:25.498601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 556/1101] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:37:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 03:37:25.504425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567688b-3e09-41ae-9314-e0ae72f9a1e1 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:25.565110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3fb89-2100-4c54-84f1-3c10ff36a11a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:25.598921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03355c2-5757-4db5-a256-090fb7b86bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:25.599720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03355c2-5757-4db5-a256-090fb7b86bf5 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:25.600023 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03355c2-5757-4db5-a256-090fb7b86bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:25.600311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03355c2-5757-4db5-a256-090fb7b86bf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:25.601456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567688b-3e09-41ae-9314-e0ae72f9a1e1 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:25.607585 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03355c2-5757-4db5-a256-090fb7b86bf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:25.609112 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03355c2-5757-4db5-a256-090fb7b86bf5 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c03355c2-5757-4db5-a256-090fb7b86bf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:37:25.623845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:25.624758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:25.625656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:37:25.625656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:37:25.625656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:37:25.625656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:37:25.625656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:37:25.625656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:37:25.625656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:37:25.625656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:25.625656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 546/1102] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:37:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:25.630942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03355c2-5757-4db5-a256-090fb7b86bf5 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:25.635227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73dca7-041c-4b35-b684-b29065420f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:25.636161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73dca7-041c-4b35-b684-b29065420f73 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:25.636453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73dca7-041c-4b35-b684-b29065420f73 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:25.636737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73dca7-041c-4b35-b684-b29065420f73 None admin] PATH_INFO: `/v3/projects/1e194449828e40c0aca9d578b203b4fc` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:25.638006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73dca7-041c-4b35-b684-b29065420f73 None admin] RBAC: Authorizing `identity:get_project(project_id=1e194449828e40c0aca9d578b203b4fc)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:25.639698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73dca7-041c-4b35-b684-b29065420f73 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:25.641765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 557/1103] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:37:25 2022] GET /identity/v3/projects/1e194449828e40c0aca9d578b203b4fc => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:25.648291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73dca7-041c-4b35-b684-b29065420f73 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:25.699794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:25.700475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:25.732581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03355c2-5757-4db5-a256-090fb7b86bf5 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:25.746576 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73dca7-041c-4b35-b684-b29065420f73 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:25.765887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7daa83-20d3-4df5-90da-82a5fd81923b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:25.766549 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7daa83-20d3-4df5-90da-82a5fd81923b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:25.766823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7daa83-20d3-4df5-90da-82a5fd81923b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:25.767094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7daa83-20d3-4df5-90da-82a5fd81923b None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:25.767872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7daa83-20d3-4df5-90da-82a5fd81923b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:25.768204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7daa83-20d3-4df5-90da-82a5fd81923b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:25.769649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7daa83-20d3-4df5-90da-82a5fd81923b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:25.779841 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43340b9-afb3-46c3-8ff9-3c9867aaed65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:25.780520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43340b9-afb3-46c3-8ff9-3c9867aaed65 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:25.780807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43340b9-afb3-46c3-8ff9-3c9867aaed65 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:25.781085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43340b9-afb3-46c3-8ff9-3c9867aaed65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:25.781523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 547/1104] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:37:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:25.781914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43340b9-afb3-46c3-8ff9-3c9867aaed65 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=1e194449828e40c0aca9d578b203b4fc, user.id=62cb6a533cb1455d8804d55acfcbc469) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:25.782229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43340b9-afb3-46c3-8ff9-3c9867aaed65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:25.783895 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43340b9-afb3-46c3-8ff9-3c9867aaed65 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:25.787522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7daa83-20d3-4df5-90da-82a5fd81923b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:25.790123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 558/1105] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:37:25 2022] GET /identity/v3/role_assignments?user.id=62cb6a533cb1455d8804d55acfcbc469&scope.project.id=1e194449828e40c0aca9d578b203b4fc&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:25.901513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7daa83-20d3-4df5-90da-82a5fd81923b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:25.932921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:25.942332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b9679-f853-41df-8513-7a9ad8426c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:25.943009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b9679-f853-41df-8513-7a9ad8426c84 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:25.943295 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b9679-f853-41df-8513-7a9ad8426c84 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:25.943581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b9679-f853-41df-8513-7a9ad8426c84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:25.944456 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b9679-f853-41df-8513-7a9ad8426c84 None admin] RBAC: Adding query filter params (role.id=eea62fc4e0174ba39b800050ebe7ff5d, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=7138318e53ff4b7885942d49b19eb0a5) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:25.944762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b9679-f853-41df-8513-7a9ad8426c84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:25.946230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b9679-f853-41df-8513-7a9ad8426c84 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:25.952986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 548/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:37:25 2022] GET /identity/v3/role_assignments?user.id=7138318e53ff4b7885942d49b19eb0a5&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=eea62fc4e0174ba39b800050ebe7ff5d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:27.832985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c73ae-e51a-4982-b655-f4dcb53d3629 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:27.833538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c73ae-e51a-4982-b655-f4dcb53d3629 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:27.833538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c73ae-e51a-4982-b655-f4dcb53d3629 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:27.835705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 559/1107] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:27.841921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47ca7d-2242-41a6-a6d0-7ffb66ac4648 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:27.842330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47ca7d-2242-41a6-a6d0-7ffb66ac4648 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:27.842526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47ca7d-2242-41a6-a6d0-7ffb66ac4648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:27.865727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5a47ca7d-2242-41a6-a6d0-7ffb66ac4648 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:27.866926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5a47ca7d-2242-41a6-a6d0-7ffb66ac4648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:27.948698 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 549/1108] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:27.973340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c73ae-e51a-4982-b655-f4dcb53d3629 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:28.084827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c73ae-e51a-4982-b655-f4dcb53d3629 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:28.101383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:28.121609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d3468-be01-4598-95a6-82ab24bd6d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:28.122419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d3468-be01-4598-95a6-82ab24bd6d5f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:28.122713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d3468-be01-4598-95a6-82ab24bd6d5f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:28.122993 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d3468-be01-4598-95a6-82ab24bd6d5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:28.124255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d3468-be01-4598-95a6-82ab24bd6d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:28.125712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240d3468-be01-4598-95a6-82ab24bd6d5f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:28.127382 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 560/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:28.133429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47ca7d-2242-41a6-a6d0-7ffb66ac4648 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:28.241893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47ca7d-2242-41a6-a6d0-7ffb66ac4648 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:28.266719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:28.279106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212e66f-b87a-4a9b-aab1-922605a816e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:28.279886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212e66f-b87a-4a9b-aab1-922605a816e6 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:28.280201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212e66f-b87a-4a9b-aab1-922605a816e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:28.280460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212e66f-b87a-4a9b-aab1-922605a816e6 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:28.281177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212e66f-b87a-4a9b-aab1-922605a816e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:28.282647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212e66f-b87a-4a9b-aab1-922605a816e6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:28.305778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 550/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:37:28 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:37:30.342118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443eb054-f077-499a-8285-c701df41e87f None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:30.342783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443eb054-f077-499a-8285-c701df41e87f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:30.342783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443eb054-f077-499a-8285-c701df41e87f None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:30.344058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 561/1111] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:30.349459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc9a3b-3af6-45f8-a8fa-a3fadc56fb25 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:30.349903 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc9a3b-3af6-45f8-a8fa-a3fadc56fb25 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:30.349998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc9a3b-3af6-45f8-a8fa-a3fadc56fb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:30.368353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-cedc9a3b-3af6-45f8-a8fa-a3fadc56fb25 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:30.369540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cedc9a3b-3af6-45f8-a8fa-a3fadc56fb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:30.438464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 551/1112] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:30.462449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443eb054-f077-499a-8285-c701df41e87f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:30.567879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443eb054-f077-499a-8285-c701df41e87f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:30.602480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc513c3b-15fd-421e-af81-97e9a1ec9b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:30.603252 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc513c3b-15fd-421e-af81-97e9a1ec9b07 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:30.603526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc513c3b-15fd-421e-af81-97e9a1ec9b07 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:30.603795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc513c3b-15fd-421e-af81-97e9a1ec9b07 None admin] PATH_INFO: `/v3/roles/ff6066b1fb8940cd9ec82c078fd8d2f6` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:30.605389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc513c3b-15fd-421e-af81-97e9a1ec9b07 None admin] RBAC: Authorizing `identity:get_role(role_id=ff6066b1fb8940cd9ec82c078fd8d2f6)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:30.607203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc513c3b-15fd-421e-af81-97e9a1ec9b07 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:30.609636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 562/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:37:30 2022] GET /identity/v3/roles/ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:30.617388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc9a3b-3af6-45f8-a8fa-a3fadc56fb25 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:30.730160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc9a3b-3af6-45f8-a8fa-a3fadc56fb25 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:30.740988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:30.767717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefc3d0-8345-49c8-a986-612bd43e2477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:30.768624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefc3d0-8345-49c8-a986-612bd43e2477 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:30.768904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefc3d0-8345-49c8-a986-612bd43e2477 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:30.769174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefc3d0-8345-49c8-a986-612bd43e2477 None admin] PATH_INFO: `/v3/users/8d3df91ccddc4e5a9e582aa8c9e2464e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:30.782178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefc3d0-8345-49c8-a986-612bd43e2477 None admin] RBAC: Authorizing `identity:get_user(user_id=8d3df91ccddc4e5a9e582aa8c9e2464e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:30.783672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefc3d0-8345-49c8-a986-612bd43e2477 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:30.785358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 552/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:37:30 2022] GET /identity/v3/users/8d3df91ccddc4e5a9e582aa8c9e2464e => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:30.791939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc513c3b-15fd-421e-af81-97e9a1ec9b07 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:30.844759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:30.893850 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc513c3b-15fd-421e-af81-97e9a1ec9b07 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:30.930420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263dd4a-d6e9-4a94-9b01-9d02047037af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:30.931158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263dd4a-d6e9-4a94-9b01-9d02047037af None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:30.931605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263dd4a-d6e9-4a94-9b01-9d02047037af None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:30.931966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263dd4a-d6e9-4a94-9b01-9d02047037af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:31.010177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263dd4a-d6e9-4a94-9b01-9d02047037af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:31.011668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263dd4a-d6e9-4a94-9b01-9d02047037af None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:31.064076 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:31.117176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 563/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:37:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:37:31.123932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefc3d0-8345-49c8-a986-612bd43e2477 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:31.250537 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefc3d0-8345-49c8-a986-612bd43e2477 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:31.290134 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecced6-e54d-4010-8b27-b6d7e9b6bec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:31.291286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecced6-e54d-4010-8b27-b6d7e9b6bec6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:31.291572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecced6-e54d-4010-8b27-b6d7e9b6bec6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:31.291845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecced6-e54d-4010-8b27-b6d7e9b6bec6 None admin] PATH_INFO: `/v3/projects/1e194449828e40c0aca9d578b203b4fc` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:31.293279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ecced6-e54d-4010-8b27-b6d7e9b6bec6 None admin] RBAC: Authorizing `identity:get_project(project_id=1e194449828e40c0aca9d578b203b4fc)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:31.295644 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ecced6-e54d-4010-8b27-b6d7e9b6bec6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:31.297416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 553/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:37:31 2022] GET /identity/v3/projects/1e194449828e40c0aca9d578b203b4fc => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:31.305970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263dd4a-d6e9-4a94-9b01-9d02047037af None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:31.426557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263dd4a-d6e9-4a94-9b01-9d02047037af None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:31.463896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059d408-75d4-4d42-b13e-d2ddedd7b21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:31.464627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059d408-75d4-4d42-b13e-d2ddedd7b21a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:31.464905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059d408-75d4-4d42-b13e-d2ddedd7b21a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:31.465179 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059d408-75d4-4d42-b13e-d2ddedd7b21a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:31.466104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059d408-75d4-4d42-b13e-d2ddedd7b21a None admin] RBAC: Adding query filter params (role.id=ff6066b1fb8940cd9ec82c078fd8d2f6, scope.project.id=1e194449828e40c0aca9d578b203b4fc, user.id=8d3df91ccddc4e5a9e582aa8c9e2464e) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:31.466416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059d408-75d4-4d42-b13e-d2ddedd7b21a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:31.467856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059d408-75d4-4d42-b13e-d2ddedd7b21a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:31.473771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 564/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:37:31 2022] GET /identity/v3/role_assignments?user.id=8d3df91ccddc4e5a9e582aa8c9e2464e&scope.project.id=1e194449828e40c0aca9d578b203b4fc&role.id=ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:33.485019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de322aa5-8993-4f34-8985-778665589c76 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:33.485920 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de322aa5-8993-4f34-8985-778665589c76 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:33.485920 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de322aa5-8993-4f34-8985-778665589c76 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:33.488073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 554/1118] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:33.495965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b597b55-b0cb-4910-8df2-51cfc347e5d3 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:33.496442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b597b55-b0cb-4910-8df2-51cfc347e5d3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:33.496538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b597b55-b0cb-4910-8df2-51cfc347e5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:33.515316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-6b597b55-b0cb-4910-8df2-51cfc347e5d3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:33.516550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6b597b55-b0cb-4910-8df2-51cfc347e5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:33.527777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:33.586020 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 565/1119] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:37:33.610821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de322aa5-8993-4f34-8985-778665589c76 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:33.658559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:33.727952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de322aa5-8993-4f34-8985-778665589c76 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:33.764106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a54f7-0d94-4241-8136-d9d8a0fc3047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:33.764915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a54f7-0d94-4241-8136-d9d8a0fc3047 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:33.765200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a54f7-0d94-4241-8136-d9d8a0fc3047 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:33.765473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a54f7-0d94-4241-8136-d9d8a0fc3047 None admin] PATH_INFO: `/v3/roles/ff6066b1fb8940cd9ec82c078fd8d2f6` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:33.767007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a54f7-0d94-4241-8136-d9d8a0fc3047 None admin] RBAC: Authorizing `identity:get_role(role_id=ff6066b1fb8940cd9ec82c078fd8d2f6)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:33.768421 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a54f7-0d94-4241-8136-d9d8a0fc3047 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:33.769786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 555/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:37:33 2022] GET /identity/v3/roles/ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:33.779488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b597b55-b0cb-4910-8df2-51cfc347e5d3 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:33.886775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b597b55-b0cb-4910-8df2-51cfc347e5d3 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:33.921658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735de92-86bf-495d-8ebb-2a5e7b324831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:33.922509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735de92-86bf-495d-8ebb-2a5e7b324831 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:33.922822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735de92-86bf-495d-8ebb-2a5e7b324831 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:33.923088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735de92-86bf-495d-8ebb-2a5e7b324831 None admin] PATH_INFO: `/v3/users/8d3df91ccddc4e5a9e582aa8c9e2464e` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:33.924308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735de92-86bf-495d-8ebb-2a5e7b324831 None admin] RBAC: Authorizing `identity:get_user(user_id=8d3df91ccddc4e5a9e582aa8c9e2464e)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:33.925754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735de92-86bf-495d-8ebb-2a5e7b324831 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:33.927361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 566/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:37:33 2022] GET /identity/v3/users/8d3df91ccddc4e5a9e582aa8c9e2464e => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:33.934109 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a54f7-0d94-4241-8136-d9d8a0fc3047 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:34.047212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a54f7-0d94-4241-8136-d9d8a0fc3047 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:34.049950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:34.087704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72048156-d978-4962-879d-b739d27ea818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:34.088605 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72048156-d978-4962-879d-b739d27ea818 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:34.088890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72048156-d978-4962-879d-b739d27ea818 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:34.089168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72048156-d978-4962-879d-b739d27ea818 None admin] PATH_INFO: `/v3/projects/1e194449828e40c0aca9d578b203b4fc` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:34.090426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72048156-d978-4962-879d-b739d27ea818 None admin] RBAC: Authorizing `identity:get_project(project_id=1e194449828e40c0aca9d578b203b4fc)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:34.091881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72048156-d978-4962-879d-b739d27ea818 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:34.093629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 556/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:37:33 2022] GET /identity/v3/projects/1e194449828e40c0aca9d578b203b4fc => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:34.100332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735de92-86bf-495d-8ebb-2a5e7b324831 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:34.117475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:34.209418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735de92-86bf-495d-8ebb-2a5e7b324831 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:34.244917 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6c2d1-b6c6-4c1e-8250-eadc9c0655c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:34.245796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6c2d1-b6c6-4c1e-8250-eadc9c0655c3 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:34.246084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6c2d1-b6c6-4c1e-8250-eadc9c0655c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:34.246356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6c2d1-b6c6-4c1e-8250-eadc9c0655c3 None admin] PATH_INFO: `/v3/projects/1e194449828e40c0aca9d578b203b4fc/users/8d3df91ccddc4e5a9e582aa8c9e2464e/roles/ff6066b1fb8940cd9ec82c078fd8d2f6` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:34.249080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6c2d1-b6c6-4c1e-8250-eadc9c0655c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e194449828e40c0aca9d578b203b4fc, user_id=8d3df91ccddc4e5a9e582aa8c9e2464e, role_id=ff6066b1fb8940cd9ec82c078fd8d2f6)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:34.250522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6c2d1-b6c6-4c1e-8250-eadc9c0655c3 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:34.265619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 567/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:37:34 2022] PUT /identity/v3/projects/1e194449828e40c0aca9d578b203b4fc/users/8d3df91ccddc4e5a9e582aa8c9e2464e/roles/ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:37:36.246797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f41f1-dba9-405c-a68f-61a11c04cc9e None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:36.247372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f41f1-dba9-405c-a68f-61a11c04cc9e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:36.247372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f41f1-dba9-405c-a68f-61a11c04cc9e None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:36.248723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 557/1124] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:36.255301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccb5b1-3252-421f-87f5-72cdebb6084c None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:36.255997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccb5b1-3252-421f-87f5-72cdebb6084c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:36.256167 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccb5b1-3252-421f-87f5-72cdebb6084c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:36.277281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-80ccb5b1-3252-421f-87f5-72cdebb6084c None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:36.278489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-80ccb5b1-3252-421f-87f5-72cdebb6084c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:36.348102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 568/1125] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:36.381688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f41f1-dba9-405c-a68f-61a11c04cc9e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:36.487466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f41f1-dba9-405c-a68f-61a11c04cc9e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:36.527298 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224d962-3d5d-4991-b050-8febfd815ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:36.528128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224d962-3d5d-4991-b050-8febfd815ae9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:36.528416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224d962-3d5d-4991-b050-8febfd815ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:36.528781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224d962-3d5d-4991-b050-8febfd815ae9 None admin] PATH_INFO: `/v3/roles/ff6066b1fb8940cd9ec82c078fd8d2f6` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:36.530409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224d962-3d5d-4991-b050-8febfd815ae9 None admin] RBAC: Authorizing `identity:get_role(role_id=ff6066b1fb8940cd9ec82c078fd8d2f6)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:36.531855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224d962-3d5d-4991-b050-8febfd815ae9 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:36.533249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 558/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:37:36 2022] GET /identity/v3/roles/ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:36.539381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccb5b1-3252-421f-87f5-72cdebb6084c None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:36.542773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:36.643431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccb5b1-3252-421f-87f5-72cdebb6084c None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:36.678621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c694982-60b7-4665-801a-6cd617003726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:36.679490 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c694982-60b7-4665-801a-6cd617003726 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:36.679772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c694982-60b7-4665-801a-6cd617003726 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:36.680047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c694982-60b7-4665-801a-6cd617003726 None admin] PATH_INFO: `/v3/users/8d3df91ccddc4e5a9e582aa8c9e2464e` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:36.681364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c694982-60b7-4665-801a-6cd617003726 None admin] RBAC: Authorizing `identity:get_user(user_id=8d3df91ccddc4e5a9e582aa8c9e2464e)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:36.682837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c694982-60b7-4665-801a-6cd617003726 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:36.684539 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 569/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:37:36 2022] GET /identity/v3/users/8d3df91ccddc4e5a9e582aa8c9e2464e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:36.690658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224d962-3d5d-4991-b050-8febfd815ae9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:36.733655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:36.805771 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224d962-3d5d-4991-b050-8febfd815ae9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:36.841290 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9c5e8-7b05-4e64-a194-3f3bb0ab9f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:36.841935 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9c5e8-7b05-4e64-a194-3f3bb0ab9f43 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:36.842226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9c5e8-7b05-4e64-a194-3f3bb0ab9f43 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:36.842504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9c5e8-7b05-4e64-a194-3f3bb0ab9f43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:36.922594 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9c5e8-7b05-4e64-a194-3f3bb0ab9f43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:36.924102 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9c5e8-7b05-4e64-a194-3f3bb0ab9f43 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:36.965473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:37.026034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 559/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:37:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:37:37.032741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c694982-60b7-4665-801a-6cd617003726 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:37.115041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:37.157994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c694982-60b7-4665-801a-6cd617003726 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:37.192601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ceabc3-e698-4a14-be04-6d7fc9334a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:37.193453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ceabc3-e698-4a14-be04-6d7fc9334a02 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:37.193734 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ceabc3-e698-4a14-be04-6d7fc9334a02 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:37.194003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ceabc3-e698-4a14-be04-6d7fc9334a02 None admin] PATH_INFO: `/v3/projects/1e194449828e40c0aca9d578b203b4fc` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:37.195212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ceabc3-e698-4a14-be04-6d7fc9334a02 None admin] RBAC: Authorizing `identity:get_project(project_id=1e194449828e40c0aca9d578b203b4fc)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:37.196700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ceabc3-e698-4a14-be04-6d7fc9334a02 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:37.198436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 570/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:37:37 2022] GET /identity/v3/projects/1e194449828e40c0aca9d578b203b4fc => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:37.204888 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9c5e8-7b05-4e64-a194-3f3bb0ab9f43 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:37.308316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9c5e8-7b05-4e64-a194-3f3bb0ab9f43 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:37.343837 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645c387-821f-4682-8776-5f91ee96b981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:37.344523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645c387-821f-4682-8776-5f91ee96b981 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:37.344805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645c387-821f-4682-8776-5f91ee96b981 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:37.345078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645c387-821f-4682-8776-5f91ee96b981 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:37.345924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645c387-821f-4682-8776-5f91ee96b981 None admin] RBAC: Adding query filter params (role.id=ff6066b1fb8940cd9ec82c078fd8d2f6, scope.project.id=1e194449828e40c0aca9d578b203b4fc, user.id=8d3df91ccddc4e5a9e582aa8c9e2464e) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:37.346234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645c387-821f-4682-8776-5f91ee96b981 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:37.347708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645c387-821f-4682-8776-5f91ee96b981 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:37.354490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 560/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:37:37 2022] GET /identity/v3/role_assignments?user.id=8d3df91ccddc4e5a9e582aa8c9e2464e&scope.project.id=1e194449828e40c0aca9d578b203b4fc&role.id=ff6066b1fb8940cd9ec82c078fd8d2f6 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:39.386639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b87a8-d88a-4a67-8c78-889171f4c679 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:39.387195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b87a8-d88a-4a67-8c78-889171f4c679 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:39.387195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b87a8-d88a-4a67-8c78-889171f4c679 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:39.388646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 571/1131] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:39.394917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3432155-d91f-4bc9-b26b-aa420e37d700 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:39.395685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3432155-d91f-4bc9-b26b-aa420e37d700 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:39.395817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3432155-d91f-4bc9-b26b-aa420e37d700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:39.405607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:39.417130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-f3432155-d91f-4bc9-b26b-aa420e37d700 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:39.418330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f3432155-d91f-4bc9-b26b-aa420e37d700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:39.487756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 561/1132] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:39.515644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b87a8-d88a-4a67-8c78-889171f4c679 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:39.624361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b87a8-d88a-4a67-8c78-889171f4c679 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:39.663131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa1d21-d39a-4b25-b685-4d80e65ade5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:39.663850 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa1d21-d39a-4b25-b685-4d80e65ade5f None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:39.664172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa1d21-d39a-4b25-b685-4d80e65ade5f None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:39.664450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa1d21-d39a-4b25-b685-4d80e65ade5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:39.665654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa1d21-d39a-4b25-b685-4d80e65ade5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:39.667101 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa1d21-d39a-4b25-b685-4d80e65ade5f None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:39.668783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 572/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:39.674761 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3432155-d91f-4bc9-b26b-aa420e37d700 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:39.780295 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3432155-d91f-4bc9-b26b-aa420e37d700 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:39.815014 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b41de-8be4-4a35-9736-459449f7ba36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:39.815671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b41de-8be4-4a35-9736-459449f7ba36 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:39.815953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b41de-8be4-4a35-9736-459449f7ba36 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:39.816253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b41de-8be4-4a35-9736-459449f7ba36 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:39.816954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b41de-8be4-4a35-9736-459449f7ba36 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:39.818407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b41de-8be4-4a35-9736-459449f7ba36 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:39.822064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:39.847087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 562/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:37:39 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:37:41.923482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1978eb-a9fe-44c1-8303-4bc5be24dc41 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:41.924100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1978eb-a9fe-44c1-8303-4bc5be24dc41 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:41.924100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1978eb-a9fe-44c1-8303-4bc5be24dc41 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:41.925448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 573/1135] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:41.932280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de1449-93cc-41c3-a92c-1af1f58dbfef None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:41.933083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de1449-93cc-41c3-a92c-1af1f58dbfef None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:41.933245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de1449-93cc-41c3-a92c-1af1f58dbfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:41.954073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-68de1449-93cc-41c3-a92c-1af1f58dbfef None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:41.955289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-68de1449-93cc-41c3-a92c-1af1f58dbfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:42.027224 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 563/1136] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:42.053175 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1978eb-a9fe-44c1-8303-4bc5be24dc41 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:42.064725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:42.163794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1978eb-a9fe-44c1-8303-4bc5be24dc41 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:42.199360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e04277-9f4f-43c7-84f2-3065b158dc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:42.200110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e04277-9f4f-43c7-84f2-3065b158dc7e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:42.200392 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e04277-9f4f-43c7-84f2-3065b158dc7e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:42.200725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e04277-9f4f-43c7-84f2-3065b158dc7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:42.201936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e04277-9f4f-43c7-84f2-3065b158dc7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:42.203381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e04277-9f4f-43c7-84f2-3065b158dc7e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:42.205066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 574/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:37:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:42.211036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de1449-93cc-41c3-a92c-1af1f58dbfef None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:42.313743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de1449-93cc-41c3-a92c-1af1f58dbfef None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:42.330963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:42.353232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d981b-8482-493c-96ef-b2c49c9e3c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:42.353948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d981b-8482-493c-96ef-b2c49c9e3c10 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:42.354230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d981b-8482-493c-96ef-b2c49c9e3c10 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:42.354514 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d981b-8482-493c-96ef-b2c49c9e3c10 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:42.355234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d981b-8482-493c-96ef-b2c49c9e3c10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:42.356748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d981b-8482-493c-96ef-b2c49c9e3c10 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:42.380793 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 564/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:37:42 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:37:44.729861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0588201-ff62-4e8d-a842-dc41f737e134 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:44.730432 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0588201-ff62-4e8d-a842-dc41f737e134 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:44.730432 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0588201-ff62-4e8d-a842-dc41f737e134 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:44.731797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 575/1139] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:37:44.738188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfd10a-836a-4a76-ba43-99ec68ea7722 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:44.738954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfd10a-836a-4a76-ba43-99ec68ea7722 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:44.739083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfd10a-836a-4a76-ba43-99ec68ea7722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:44.761323 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dbdfd10a-836a-4a76-ba43-99ec68ea7722 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:44.762521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dbdfd10a-836a-4a76-ba43-99ec68ea7722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:44.833666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 565/1140] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:44.858128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0588201-ff62-4e8d-a842-dc41f737e134 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:44.959326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:44.988195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0588201-ff62-4e8d-a842-dc41f737e134 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:45.025218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48311852-af57-446e-9cb9-90f4205244d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:45.026012 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48311852-af57-446e-9cb9-90f4205244d8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:45.026295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48311852-af57-446e-9cb9-90f4205244d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:45.026567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48311852-af57-446e-9cb9-90f4205244d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:45.037201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48311852-af57-446e-9cb9-90f4205244d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-48311852-af57-446e-9cb9-90f4205244d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:37:45.044891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:45.046000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:45.046920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 576/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:37:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:45.054331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfd10a-836a-4a76-ba43-99ec68ea7722 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:45.184895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfd10a-836a-4a76-ba43-99ec68ea7722 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:45.186755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:45.223647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f8b7b-7c2b-4a39-b5f3-83f43d5c7d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:45.224352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f8b7b-7c2b-4a39-b5f3-83f43d5c7d37 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:45.224639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f8b7b-7c2b-4a39-b5f3-83f43d5c7d37 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:45.224915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f8b7b-7c2b-4a39-b5f3-83f43d5c7d37 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:45.225693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f8b7b-7c2b-4a39-b5f3-83f43d5c7d37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:45.226000 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f8b7b-7c2b-4a39-b5f3-83f43d5c7d37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:45.227393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f8b7b-7c2b-4a39-b5f3-83f43d5c7d37 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:45.236381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 566/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:37:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:45.246028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48311852-af57-446e-9cb9-90f4205244d8 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:45.355904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48311852-af57-446e-9cb9-90f4205244d8 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:45.392863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b80078-be7b-44d4-8904-b1e517c24890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:45.393716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b80078-be7b-44d4-8904-b1e517c24890 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:45.393998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b80078-be7b-44d4-8904-b1e517c24890 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:45.394267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b80078-be7b-44d4-8904-b1e517c24890 None admin] PATH_INFO: `/v3/users/fc8c7d3236d944458f90ab8fc6c8dd74` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:45.397088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:45.412043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b80078-be7b-44d4-8904-b1e517c24890 None admin] RBAC: Authorizing `identity:get_user(user_id=fc8c7d3236d944458f90ab8fc6c8dd74)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:45.413524 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b80078-be7b-44d4-8904-b1e517c24890 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:45.415212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 577/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:37:45 2022] GET /identity/v3/users/fc8c7d3236d944458f90ab8fc6c8dd74 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:45.421198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f8b7b-7c2b-4a39-b5f3-83f43d5c7d37 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:45.529235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f8b7b-7c2b-4a39-b5f3-83f43d5c7d37 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:45.564740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24f26b-392d-49bb-bcf4-2ee872b2db15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:45.565389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24f26b-392d-49bb-bcf4-2ee872b2db15 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:45.565671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24f26b-392d-49bb-bcf4-2ee872b2db15 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:45.565946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24f26b-392d-49bb-bcf4-2ee872b2db15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:45.597268 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:45.638091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24f26b-392d-49bb-bcf4-2ee872b2db15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:45.639566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24f26b-392d-49bb-bcf4-2ee872b2db15 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:45.743392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 567/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:37:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:37:45.749604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b80078-be7b-44d4-8904-b1e517c24890 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:45.852473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b80078-be7b-44d4-8904-b1e517c24890 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:45.888799 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f0fe7-7568-4f3c-8095-33948ef0c0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:45.889654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f0fe7-7568-4f3c-8095-33948ef0c0bb None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:45.889938 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f0fe7-7568-4f3c-8095-33948ef0c0bb None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:45.890210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f0fe7-7568-4f3c-8095-33948ef0c0bb None admin] PATH_INFO: `/v3/projects/de3784612bd94164af6904bfe558a473` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:45.891502 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f0fe7-7568-4f3c-8095-33948ef0c0bb None admin] RBAC: Authorizing `identity:get_project(project_id=de3784612bd94164af6904bfe558a473)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:45.892981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f0fe7-7568-4f3c-8095-33948ef0c0bb None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:45.894669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 578/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:37:45 2022] GET /identity/v3/projects/de3784612bd94164af6904bfe558a473 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:45.900952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24f26b-392d-49bb-bcf4-2ee872b2db15 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:45.968601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:46.007212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24f26b-392d-49bb-bcf4-2ee872b2db15 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:46.042776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a143a14-ee54-4f2f-8e30-8587a1fecf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:46.043435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a143a14-ee54-4f2f-8e30-8587a1fecf99 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:46.043719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a143a14-ee54-4f2f-8e30-8587a1fecf99 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:46.043993 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a143a14-ee54-4f2f-8e30-8587a1fecf99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:46.044917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a143a14-ee54-4f2f-8e30-8587a1fecf99 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=de3784612bd94164af6904bfe558a473, user.id=fc8c7d3236d944458f90ab8fc6c8dd74) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:46.045230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a143a14-ee54-4f2f-8e30-8587a1fecf99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:46.046668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a143a14-ee54-4f2f-8e30-8587a1fecf99 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:46.052202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 568/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:37:45 2022] GET /identity/v3/role_assignments?user.id=fc8c7d3236d944458f90ab8fc6c8dd74&scope.project.id=de3784612bd94164af6904bfe558a473&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:37:48.052064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a08e74-2b30-4c32-904a-e8ad88f997ac None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:48.052901 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a08e74-2b30-4c32-904a-e8ad88f997ac None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:48.053011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a08e74-2b30-4c32-904a-e8ad88f997ac None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:48.055231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 579/1147] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:48.061659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b19ce-7708-40db-a46d-9406fc39d226 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:48.062119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b19ce-7708-40db-a46d-9406fc39d226 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:48.062212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b19ce-7708-40db-a46d-9406fc39d226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:48.085502 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-989b19ce-7708-40db-a46d-9406fc39d226 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:48.086683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-989b19ce-7708-40db-a46d-9406fc39d226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:48.157399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 569/1148] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:48.179750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a08e74-2b30-4c32-904a-e8ad88f997ac None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:48.254398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:48.294970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a08e74-2b30-4c32-904a-e8ad88f997ac None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:48.329765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9138388-19f2-4367-b4b0-29fc13d0480d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:48.330542 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9138388-19f2-4367-b4b0-29fc13d0480d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:48.330822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9138388-19f2-4367-b4b0-29fc13d0480d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:48.331099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9138388-19f2-4367-b4b0-29fc13d0480d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:48.342144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9138388-19f2-4367-b4b0-29fc13d0480d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-b9138388-19f2-4367-b4b0-29fc13d0480d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:37:48.349677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:48.350732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:48.351567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 580/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:37:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:48.359734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b19ce-7708-40db-a46d-9406fc39d226 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:48.429853 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:48.473810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b19ce-7708-40db-a46d-9406fc39d226 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:48.517061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029829e-468e-40ef-bffa-5c1affe8c76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:48.517925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029829e-468e-40ef-bffa-5c1affe8c76b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:48.518300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029829e-468e-40ef-bffa-5c1affe8c76b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:48.518639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029829e-468e-40ef-bffa-5c1affe8c76b None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:48.519856 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029829e-468e-40ef-bffa-5c1affe8c76b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:48.520290 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029829e-468e-40ef-bffa-5c1affe8c76b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:48.522478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029829e-468e-40ef-bffa-5c1affe8c76b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:48.533561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 570/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:37:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:48.539743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9138388-19f2-4367-b4b0-29fc13d0480d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:48.640283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9138388-19f2-4367-b4b0-29fc13d0480d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:48.662828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:48.683486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd14be-db66-4fac-a68a-df85c5940806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:48.684799 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd14be-db66-4fac-a68a-df85c5940806 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:48.685237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd14be-db66-4fac-a68a-df85c5940806 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:48.685520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bd14be-db66-4fac-a68a-df85c5940806 None admin] PATH_INFO: `/v3/users/fc8c7d3236d944458f90ab8fc6c8dd74` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:48.686737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd14be-db66-4fac-a68a-df85c5940806 None admin] RBAC: Authorizing `identity:get_user(user_id=fc8c7d3236d944458f90ab8fc6c8dd74)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:48.688218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bd14be-db66-4fac-a68a-df85c5940806 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:48.689864 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 581/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:37:48 2022] GET /identity/v3/users/fc8c7d3236d944458f90ab8fc6c8dd74 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:48.696048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029829e-468e-40ef-bffa-5c1affe8c76b None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:48.803031 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029829e-468e-40ef-bffa-5c1affe8c76b None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:48.827857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:48.846955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178517d6-d438-448c-9d3c-c34a59e1d1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:48.848330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178517d6-d438-448c-9d3c-c34a59e1d1aa None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:48.848665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178517d6-d438-448c-9d3c-c34a59e1d1aa None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:48.849030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178517d6-d438-448c-9d3c-c34a59e1d1aa None admin] PATH_INFO: `/v3/projects/de3784612bd94164af6904bfe558a473` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:48.851028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178517d6-d438-448c-9d3c-c34a59e1d1aa None admin] RBAC: Authorizing `identity:get_project(project_id=de3784612bd94164af6904bfe558a473)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:48.853355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178517d6-d438-448c-9d3c-c34a59e1d1aa None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:48.855942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 571/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:37:48 2022] GET /identity/v3/projects/de3784612bd94164af6904bfe558a473 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:48.863725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd14be-db66-4fac-a68a-df85c5940806 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:48.968182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bd14be-db66-4fac-a68a-df85c5940806 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:49.002549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e4156-9497-4ee0-9f65-f9b059aafbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:49.003426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e4156-9497-4ee0-9f65-f9b059aafbac None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:49.003706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e4156-9497-4ee0-9f65-f9b059aafbac None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:49.003978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e4156-9497-4ee0-9f65-f9b059aafbac None admin] PATH_INFO: `/v3/projects/de3784612bd94164af6904bfe558a473/users/fc8c7d3236d944458f90ab8fc6c8dd74/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:49.006044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e4156-9497-4ee0-9f65-f9b059aafbac None admin] RBAC: Authorizing `identity:create_grant(project_id=de3784612bd94164af6904bfe558a473, user_id=fc8c7d3236d944458f90ab8fc6c8dd74, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:49.007477 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e4156-9497-4ee0-9f65-f9b059aafbac None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:49.023611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 582/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:37:48 2022] PUT /identity/v3/projects/de3784612bd94164af6904bfe558a473/users/fc8c7d3236d944458f90ab8fc6c8dd74/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:37:51.000341 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fca6a8-624b-4fd1-810f-dbf2b378cb3a None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:51.001151 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fca6a8-624b-4fd1-810f-dbf2b378cb3a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:51.001289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fca6a8-624b-4fd1-810f-dbf2b378cb3a None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:51.003468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 572/1154] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:51.012478 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd9080-57cd-4bb5-832b-0ddedf56e06b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:51.012889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd9080-57cd-4bb5-832b-0ddedf56e06b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:51.013105 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd9080-57cd-4bb5-832b-0ddedf56e06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:51.031113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a3cd9080-57cd-4bb5-832b-0ddedf56e06b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:51.032354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a3cd9080-57cd-4bb5-832b-0ddedf56e06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:51.080418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:51.112581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 583/1155] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:51.136844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fca6a8-624b-4fd1-810f-dbf2b378cb3a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:51.253805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fca6a8-624b-4fd1-810f-dbf2b378cb3a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:51.289561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc448dd4-66b2-4879-b35d-015c5d992b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:51.290349 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc448dd4-66b2-4879-b35d-015c5d992b82 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:51.290635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc448dd4-66b2-4879-b35d-015c5d992b82 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:51.290912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc448dd4-66b2-4879-b35d-015c5d992b82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:51.302498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc448dd4-66b2-4879-b35d-015c5d992b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-cc448dd4-66b2-4879-b35d-015c5d992b82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:37:51.310099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:51.311205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:37:51.312108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 573/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:37:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:51.320581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd9080-57cd-4bb5-832b-0ddedf56e06b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:51.434012 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd9080-57cd-4bb5-832b-0ddedf56e06b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:51.469227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199da7c-d61d-4032-a566-6eb4225af7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:51.469880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199da7c-d61d-4032-a566-6eb4225af7cc None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:51.470164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199da7c-d61d-4032-a566-6eb4225af7cc None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:51.470437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199da7c-d61d-4032-a566-6eb4225af7cc None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:51.471179 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199da7c-d61d-4032-a566-6eb4225af7cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:51.471481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199da7c-d61d-4032-a566-6eb4225af7cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:51.472897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1199da7c-d61d-4032-a566-6eb4225af7cc None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:51.481314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 584/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:37:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:51.487801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc448dd4-66b2-4879-b35d-015c5d992b82 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:51.549149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:51.604170 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc448dd4-66b2-4879-b35d-015c5d992b82 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:51.640310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af78f2a-106e-43c7-9d3d-42e788515938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:51.641202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af78f2a-106e-43c7-9d3d-42e788515938 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:51.641524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af78f2a-106e-43c7-9d3d-42e788515938 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:51.641791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af78f2a-106e-43c7-9d3d-42e788515938 None admin] PATH_INFO: `/v3/users/fc8c7d3236d944458f90ab8fc6c8dd74` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:51.643505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af78f2a-106e-43c7-9d3d-42e788515938 None admin] RBAC: Authorizing `identity:get_user(user_id=fc8c7d3236d944458f90ab8fc6c8dd74)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:51.645507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af78f2a-106e-43c7-9d3d-42e788515938 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:51.647702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 574/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:37:51 2022] GET /identity/v3/users/fc8c7d3236d944458f90ab8fc6c8dd74 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:51.656347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199da7c-d61d-4032-a566-6eb4225af7cc None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:51.701921 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:51.767340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199da7c-d61d-4032-a566-6eb4225af7cc None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:51.813838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21314dff-497b-4bfa-8e29-eb2067400fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:51.814489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21314dff-497b-4bfa-8e29-eb2067400fa3 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:51.814777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21314dff-497b-4bfa-8e29-eb2067400fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:51.815056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21314dff-497b-4bfa-8e29-eb2067400fa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:51.892199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21314dff-497b-4bfa-8e29-eb2067400fa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:51.893727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21314dff-497b-4bfa-8e29-eb2067400fa3 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:51.938326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:52.001128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 585/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:37:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:37:52.008382 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af78f2a-106e-43c7-9d3d-42e788515938 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:52.114343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af78f2a-106e-43c7-9d3d-42e788515938 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:52.130902 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:52.150110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35468625-9577-488c-b5e5-0db319fdaab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:52.151012 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35468625-9577-488c-b5e5-0db319fdaab7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:52.151292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35468625-9577-488c-b5e5-0db319fdaab7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:52.151579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35468625-9577-488c-b5e5-0db319fdaab7 None admin] PATH_INFO: `/v3/projects/de3784612bd94164af6904bfe558a473` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:52.152853 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35468625-9577-488c-b5e5-0db319fdaab7 None admin] RBAC: Authorizing `identity:get_project(project_id=de3784612bd94164af6904bfe558a473)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:52.154387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35468625-9577-488c-b5e5-0db319fdaab7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:52.156144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 575/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:37:52 2022] GET /identity/v3/projects/de3784612bd94164af6904bfe558a473 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:52.163158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21314dff-497b-4bfa-8e29-eb2067400fa3 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:52.270546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21314dff-497b-4bfa-8e29-eb2067400fa3 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:52.307382 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48e0c7-d39e-4d76-b4ff-2a6992b81881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:52.308049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48e0c7-d39e-4d76-b4ff-2a6992b81881 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:52.308325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48e0c7-d39e-4d76-b4ff-2a6992b81881 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:52.308596 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48e0c7-d39e-4d76-b4ff-2a6992b81881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:52.309430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48e0c7-d39e-4d76-b4ff-2a6992b81881 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=de3784612bd94164af6904bfe558a473, user.id=fc8c7d3236d944458f90ab8fc6c8dd74) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:52.309739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48e0c7-d39e-4d76-b4ff-2a6992b81881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:52.311341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48e0c7-d39e-4d76-b4ff-2a6992b81881 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:52.317331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 586/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:37:52 2022] GET /identity/v3/role_assignments?user.id=fc8c7d3236d944458f90ab8fc6c8dd74&scope.project.id=de3784612bd94164af6904bfe558a473&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:54.328051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e0651-0717-4fb9-b068-02e583badd5c None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:54.328623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e0651-0717-4fb9-b068-02e583badd5c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:54.328623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e0651-0717-4fb9-b068-02e583badd5c None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:54.329981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 576/1162] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:54.336307 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea295a-2b29-4328-9de6-5dc238da7274 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:54.337021 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea295a-2b29-4328-9de6-5dc238da7274 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:54.337238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea295a-2b29-4328-9de6-5dc238da7274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:54.353170 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:54.359899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d4ea295a-2b29-4328-9de6-5dc238da7274 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:54.361176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d4ea295a-2b29-4328-9de6-5dc238da7274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:54.431645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 587/1163] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:54.455008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e0651-0717-4fb9-b068-02e583badd5c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:54.570620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e0651-0717-4fb9-b068-02e583badd5c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:54.605817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae931f4-50ad-4b41-a1e9-9bf886baf35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:54.606383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae931f4-50ad-4b41-a1e9-9bf886baf35a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:54.606660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae931f4-50ad-4b41-a1e9-9bf886baf35a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:54.606930 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae931f4-50ad-4b41-a1e9-9bf886baf35a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:54.658614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:54.677818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae931f4-50ad-4b41-a1e9-9bf886baf35a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:54.679313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae931f4-50ad-4b41-a1e9-9bf886baf35a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:54.779059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 577/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:37:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:37:54.788036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea295a-2b29-4328-9de6-5dc238da7274 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:54.893822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea295a-2b29-4328-9de6-5dc238da7274 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:54.928693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75647eaa-364d-4047-8764-884ebf5f2ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:54.929480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75647eaa-364d-4047-8764-884ebf5f2ad7 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:54.929763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75647eaa-364d-4047-8764-884ebf5f2ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:54.930054 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75647eaa-364d-4047-8764-884ebf5f2ad7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:54.933057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:54.938765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75647eaa-364d-4047-8764-884ebf5f2ad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:54.940556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75647eaa-364d-4047-8764-884ebf5f2ad7 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-75647eaa-364d-4047-8764-884ebf5f2ad7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:37:54.955009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:37:54.956188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:37:54.957048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 588/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 12 03:37:54 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:37:54.962454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae931f4-50ad-4b41-a1e9-9bf886baf35a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:55.044153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:55.061048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae931f4-50ad-4b41-a1e9-9bf886baf35a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:55.096414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b58836-f835-4996-91a8-706a318badae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:55.097088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b58836-f835-4996-91a8-706a318badae None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:55.097381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b58836-f835-4996-91a8-706a318badae None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:55.097655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b58836-f835-4996-91a8-706a318badae None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:55.098387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b58836-f835-4996-91a8-706a318badae None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:37:55.098694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b58836-f835-4996-91a8-706a318badae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:55.100114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b58836-f835-4996-91a8-706a318badae None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:55.107414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 578/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Tue Jul 12 03:37:54 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:57.098136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65f79e-0056-4feb-9707-aa3e13668177 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:57.098698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65f79e-0056-4feb-9707-aa3e13668177 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:57.098698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65f79e-0056-4feb-9707-aa3e13668177 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:57.100111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 589/1167] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:57.106563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79bd8e-0867-472b-8e20-435b949c09f6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:57.107298 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79bd8e-0867-472b-8e20-435b949c09f6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:57.107433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79bd8e-0867-472b-8e20-435b949c09f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:57.126791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-bc79bd8e-0867-472b-8e20-435b949c09f6 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:57.127964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bc79bd8e-0867-472b-8e20-435b949c09f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:57.199783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 579/1168] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:57.224631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65f79e-0056-4feb-9707-aa3e13668177 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:57.337013 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65f79e-0056-4feb-9707-aa3e13668177 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:57.376882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa46ff8-9284-49c8-94d5-4b9c509fb5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:57.377523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa46ff8-9284-49c8-94d5-4b9c509fb5e3 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:57.377802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa46ff8-9284-49c8-94d5-4b9c509fb5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:57.378071 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa46ff8-9284-49c8-94d5-4b9c509fb5e3 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:57.378797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa46ff8-9284-49c8-94d5-4b9c509fb5e3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:57.380217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa46ff8-9284-49c8-94d5-4b9c509fb5e3 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:57.397152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 590/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Tue Jul 12 03:37:57 2022] POST /identity/v3/domains => generated 267 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:37:59.428816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3769e1-e3f6-477c-b139-20d86d8698c5 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:59.429375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3769e1-e3f6-477c-b139-20d86d8698c5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:59.429375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3769e1-e3f6-477c-b139-20d86d8698c5 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:59.430718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 580/1170] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:37:59.437066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd251fa-6cdd-4677-bd50-cfc8dac89c5b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:59.438140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd251fa-6cdd-4677-bd50-cfc8dac89c5b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:59.438140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd251fa-6cdd-4677-bd50-cfc8dac89c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:59.461404 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-4fd251fa-6cdd-4677-bd50-cfc8dac89c5b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:37:59.462599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4fd251fa-6cdd-4677-bd50-cfc8dac89c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:37:59.515782 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:59.537120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 591/1171] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:37:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:37:59.560969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3769e1-e3f6-477c-b139-20d86d8698c5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:59.646051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:37:59.670755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3769e1-e3f6-477c-b139-20d86d8698c5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:59.705806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1d564-a77d-4c73-8b5b-f3805400322b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:59.706587 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1d564-a77d-4c73-8b5b-f3805400322b None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:59.706872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1d564-a77d-4c73-8b5b-f3805400322b None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:59.707143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1d564-a77d-4c73-8b5b-f3805400322b None admin] PATH_INFO: `/v3/domains/b299af334fb9461cb1216553d2632473` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:59.718368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1d564-a77d-4c73-8b5b-f3805400322b None admin] RBAC: Authorizing `identity:get_domain(domain_id=b299af334fb9461cb1216553d2632473)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:59.719824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1d564-a77d-4c73-8b5b-f3805400322b None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:59.721478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 581/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Jul 12 03:37:59 2022] GET /identity/v3/domains/b299af334fb9461cb1216553d2632473 => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:37:59.730907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd251fa-6cdd-4677-bd50-cfc8dac89c5b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:37:59.837864 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd251fa-6cdd-4677-bd50-cfc8dac89c5b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:37:59.873030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e01ce-9645-46d3-aa11-da70b737cd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:37:59.873685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e01ce-9645-46d3-aa11-da70b737cd4b None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:37:59.873965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e01ce-9645-46d3-aa11-da70b737cd4b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:37:59.874241 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e01ce-9645-46d3-aa11-da70b737cd4b None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:37:59.875016 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e01ce-9645-46d3-aa11-da70b737cd4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:37:59.876525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e01ce-9645-46d3-aa11-da70b737cd4b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:37:59.904721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 592/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:37:59 2022] POST /identity/v3/projects => generated 361 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:38:01.933096 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e37bab-2bb6-4c32-bd07-202dee7418b9 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:01.933782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e37bab-2bb6-4c32-bd07-202dee7418b9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:01.933782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e37bab-2bb6-4c32-bd07-202dee7418b9 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:01.935020 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 582/1174] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:38:01.941493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8e13-4eac-4612-88d5-f42dde5b3a90 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:01.942261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8e13-4eac-4612-88d5-f42dde5b3a90 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:01.942427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd8e13-4eac-4612-88d5-f42dde5b3a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:01.969520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-aebd8e13-4eac-4612-88d5-f42dde5b3a90 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:38:01.971104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-aebd8e13-4eac-4612-88d5-f42dde5b3a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:38:02.003792 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:02.044956 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 593/1175] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:38:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:38:02.070724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e37bab-2bb6-4c32-bd07-202dee7418b9 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:02.179976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e37bab-2bb6-4c32-bd07-202dee7418b9 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:02.214208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c023c5-6831-46de-acbb-e7a8775ef2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:02.215001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c023c5-6831-46de-acbb-e7a8775ef2c1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:02.215293 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c023c5-6831-46de-acbb-e7a8775ef2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:02.215557 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c023c5-6831-46de-acbb-e7a8775ef2c1 None admin] PATH_INFO: `/v3/domains/b299af334fb9461cb1216553d2632473` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:02.216783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c023c5-6831-46de-acbb-e7a8775ef2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b299af334fb9461cb1216553d2632473)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:02.218249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c023c5-6831-46de-acbb-e7a8775ef2c1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:02.219823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 583/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Tue Jul 12 03:38:02 2022] GET /identity/v3/domains/b299af334fb9461cb1216553d2632473 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:38:02.225655 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd8e13-4eac-4612-88d5-f42dde5b3a90 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:02.333343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd8e13-4eac-4612-88d5-f42dde5b3a90 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:02.368310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7473e1b-e7aa-4608-8c9a-2d6d14a836df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:02.368966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7473e1b-e7aa-4608-8c9a-2d6d14a836df None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:02.369250 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7473e1b-e7aa-4608-8c9a-2d6d14a836df None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:02.369522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7473e1b-e7aa-4608-8c9a-2d6d14a836df None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:02.370234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7473e1b-e7aa-4608-8c9a-2d6d14a836df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:02.371682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7473e1b-e7aa-4608-8c9a-2d6d14a836df None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:02.396700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 594/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:38:02 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:38:04.455716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531390d2-84f3-4ce5-a3a9-050a836ecc67 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:04.456326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531390d2-84f3-4ce5-a3a9-050a836ecc67 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:04.456326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531390d2-84f3-4ce5-a3a9-050a836ecc67 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:04.458362 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 584/1178] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:38:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:38:04.467416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60515e-407b-4e34-853c-14f3066fc3ef None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:04.467824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60515e-407b-4e34-853c-14f3066fc3ef None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:04.468067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60515e-407b-4e34-853c-14f3066fc3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:04.488089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-4d60515e-407b-4e34-853c-14f3066fc3ef None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:38:04.489290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4d60515e-407b-4e34-853c-14f3066fc3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:38:04.506889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:04.559858 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 595/1179] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:38:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:38:04.584072 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531390d2-84f3-4ce5-a3a9-050a836ecc67 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:04.594624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:04.694559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531390d2-84f3-4ce5-a3a9-050a836ecc67 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:04.729683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c98e5-03f8-4a0b-b4df-9dce0486d4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:04.730418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c98e5-03f8-4a0b-b4df-9dce0486d4d9 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:04.730700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c98e5-03f8-4a0b-b4df-9dce0486d4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:04.730975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c98e5-03f8-4a0b-b4df-9dce0486d4d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:04.742751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c98e5-03f8-4a0b-b4df-9dce0486d4d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-753c98e5-03f8-4a0b-b4df-9dce0486d4d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:38:04.750322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:38:04.751449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:38:04.752566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 585/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:38:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:38:04.757841 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60515e-407b-4e34-853c-14f3066fc3ef None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:04.862856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60515e-407b-4e34-853c-14f3066fc3ef None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:04.899216 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e327b6c-8f79-4ec3-8d70-409050eab576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:04.899869 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e327b6c-8f79-4ec3-8d70-409050eab576 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:04.900192 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e327b6c-8f79-4ec3-8d70-409050eab576 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:04.900468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e327b6c-8f79-4ec3-8d70-409050eab576 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:04.901211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e327b6c-8f79-4ec3-8d70-409050eab576 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:38:04.901522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e327b6c-8f79-4ec3-8d70-409050eab576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:04.902914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e327b6c-8f79-4ec3-8d70-409050eab576 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:04.911084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 596/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:38:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:38:04.919991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c98e5-03f8-4a0b-b4df-9dce0486d4d9 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:04.999481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:05.047578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c98e5-03f8-4a0b-b4df-9dce0486d4d9 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:05.084439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b9701-33d0-46a4-8b4f-04e527305b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:05.085299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b9701-33d0-46a4-8b4f-04e527305b6c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:05.085580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b9701-33d0-46a4-8b4f-04e527305b6c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:05.085858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b9701-33d0-46a4-8b4f-04e527305b6c None admin] PATH_INFO: `/v3/users/89014035004941d28d9dcc25e28979c0` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:05.099690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b9701-33d0-46a4-8b4f-04e527305b6c None admin] RBAC: Authorizing `identity:get_user(user_id=89014035004941d28d9dcc25e28979c0)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:05.101178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b9701-33d0-46a4-8b4f-04e527305b6c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:05.102899 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 586/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:38:04 2022] GET /identity/v3/users/89014035004941d28d9dcc25e28979c0 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:38:05.109098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e327b6c-8f79-4ec3-8d70-409050eab576 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:05.130225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:05.215062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e327b6c-8f79-4ec3-8d70-409050eab576 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:05.251510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d7ad0-dce8-4d91-9e00-0171708019a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:05.252105 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d7ad0-dce8-4d91-9e00-0171708019a0 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:05.252396 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d7ad0-dce8-4d91-9e00-0171708019a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:05.252666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d7ad0-dce8-4d91-9e00-0171708019a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:05.322315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d7ad0-dce8-4d91-9e00-0171708019a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:05.323787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d7ad0-dce8-4d91-9e00-0171708019a0 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:05.340939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:05.427603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 597/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:38:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:38:05.433776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b9701-33d0-46a4-8b4f-04e527305b6c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:05.537286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b9701-33d0-46a4-8b4f-04e527305b6c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:05.562276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:05.575670 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c81f6-8665-407e-96f0-8253500ee5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:05.576578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c81f6-8665-407e-96f0-8253500ee5cd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:05.576859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c81f6-8665-407e-96f0-8253500ee5cd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:05.577131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c81f6-8665-407e-96f0-8253500ee5cd None admin] PATH_INFO: `/v3/projects/d1ddebe30dd94b87a5220ad6228fa27b` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:05.578444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c81f6-8665-407e-96f0-8253500ee5cd None admin] RBAC: Authorizing `identity:get_project(project_id=d1ddebe30dd94b87a5220ad6228fa27b)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:05.579883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c81f6-8665-407e-96f0-8253500ee5cd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:05.581610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 587/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:38:05 2022] GET /identity/v3/projects/d1ddebe30dd94b87a5220ad6228fa27b => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:38:05.588137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d7ad0-dce8-4d91-9e00-0171708019a0 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:05.694790 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d7ad0-dce8-4d91-9e00-0171708019a0 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:05.712579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:05.731756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ba30a-247c-4a6e-941f-21117ab303f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:05.732428 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ba30a-247c-4a6e-941f-21117ab303f3 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:05.732712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ba30a-247c-4a6e-941f-21117ab303f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:05.733009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ba30a-247c-4a6e-941f-21117ab303f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:05.733815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ba30a-247c-4a6e-941f-21117ab303f3 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=d1ddebe30dd94b87a5220ad6228fa27b, user.id=89014035004941d28d9dcc25e28979c0) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:38:05.734149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ba30a-247c-4a6e-941f-21117ab303f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:05.736395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ba30a-247c-4a6e-941f-21117ab303f3 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:05.742154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 598/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:38:05 2022] GET /identity/v3/role_assignments?user.id=89014035004941d28d9dcc25e28979c0&scope.project.id=d1ddebe30dd94b87a5220ad6228fa27b&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:38:07.718401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c38d64-4a61-4830-be4e-3e10e190e51a None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:07.719263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c38d64-4a61-4830-be4e-3e10e190e51a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:07.719263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c38d64-4a61-4830-be4e-3e10e190e51a None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:07.721446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 588/1186] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:38:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:38:07.730100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821c878-e6e5-4806-9025-3a85062b60d9 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:07.730531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821c878-e6e5-4806-9025-3a85062b60d9 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:07.730678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821c878-e6e5-4806-9025-3a85062b60d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:07.749644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-6821c878-e6e5-4806-9025-3a85062b60d9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:38:07.750827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6821c878-e6e5-4806-9025-3a85062b60d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:38:07.824735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 599/1187] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:38:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:38:07.854627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c38d64-4a61-4830-be4e-3e10e190e51a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:07.960383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c38d64-4a61-4830-be4e-3e10e190e51a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:07.995483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab2545-72ce-4de8-b421-9f26c2b7a955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:07.996220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab2545-72ce-4de8-b421-9f26c2b7a955 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:07.996505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab2545-72ce-4de8-b421-9f26c2b7a955 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:07.996774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab2545-72ce-4de8-b421-9f26c2b7a955 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:08.006782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab2545-72ce-4de8-b421-9f26c2b7a955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-3aab2545-72ce-4de8-b421-9f26c2b7a955 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:38:08.014603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:38:08.015660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:38:08.016583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 589/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:38:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:38:08.025310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821c878-e6e5-4806-9025-3a85062b60d9 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:08.130056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821c878-e6e5-4806-9025-3a85062b60d9 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:08.140927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:08.170565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508b9aa-7f22-4df8-b43a-01416ef8678b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:08.171232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508b9aa-7f22-4df8-b43a-01416ef8678b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:08.171576 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508b9aa-7f22-4df8-b43a-01416ef8678b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:08.171859 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508b9aa-7f22-4df8-b43a-01416ef8678b None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:08.172646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508b9aa-7f22-4df8-b43a-01416ef8678b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:38:08.172957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508b9aa-7f22-4df8-b43a-01416ef8678b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:08.174340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508b9aa-7f22-4df8-b43a-01416ef8678b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:08.182537 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 600/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:38:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:38:08.191706 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab2545-72ce-4de8-b421-9f26c2b7a955 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:08.249545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:08.300087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab2545-72ce-4de8-b421-9f26c2b7a955 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:08.335566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392d206-341b-4fae-8fdd-24ba4ac6ae3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:08.336456 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392d206-341b-4fae-8fdd-24ba4ac6ae3a None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:08.336740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392d206-341b-4fae-8fdd-24ba4ac6ae3a None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:08.337052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392d206-341b-4fae-8fdd-24ba4ac6ae3a None admin] PATH_INFO: `/v3/users/89014035004941d28d9dcc25e28979c0` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:08.338418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392d206-341b-4fae-8fdd-24ba4ac6ae3a None admin] RBAC: Authorizing `identity:get_user(user_id=89014035004941d28d9dcc25e28979c0)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:08.339851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392d206-341b-4fae-8fdd-24ba4ac6ae3a None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:08.341485 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 590/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:38:08 2022] GET /identity/v3/users/89014035004941d28d9dcc25e28979c0 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:38:08.347814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508b9aa-7f22-4df8-b43a-01416ef8678b None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:08.460024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508b9aa-7f22-4df8-b43a-01416ef8678b None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:08.495106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e70c2-b2c3-4411-bb3f-0116cbaeb5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:08.495962 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e70c2-b2c3-4411-bb3f-0116cbaeb5aa None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:08.496282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e70c2-b2c3-4411-bb3f-0116cbaeb5aa None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:08.496562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e70c2-b2c3-4411-bb3f-0116cbaeb5aa None admin] PATH_INFO: `/v3/projects/d1ddebe30dd94b87a5220ad6228fa27b` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:08.497818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e70c2-b2c3-4411-bb3f-0116cbaeb5aa None admin] RBAC: Authorizing `identity:get_project(project_id=d1ddebe30dd94b87a5220ad6228fa27b)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:08.499298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e70c2-b2c3-4411-bb3f-0116cbaeb5aa None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:08.501141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 601/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:38:08 2022] GET /identity/v3/projects/d1ddebe30dd94b87a5220ad6228fa27b => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:38:08.511008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392d206-341b-4fae-8fdd-24ba4ac6ae3a None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:08.615345 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392d206-341b-4fae-8fdd-24ba4ac6ae3a None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:08.636154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:08.651474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085e110-3781-442b-a256-73db9752e6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:08.652375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085e110-3781-442b-a256-73db9752e6ad None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:08.652654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085e110-3781-442b-a256-73db9752e6ad None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:08.652918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085e110-3781-442b-a256-73db9752e6ad None admin] PATH_INFO: `/v3/projects/d1ddebe30dd94b87a5220ad6228fa27b/users/89014035004941d28d9dcc25e28979c0/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:08.655136 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085e110-3781-442b-a256-73db9752e6ad None admin] RBAC: Authorizing `identity:create_grant(project_id=d1ddebe30dd94b87a5220ad6228fa27b, user_id=89014035004941d28d9dcc25e28979c0, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:08.657544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085e110-3781-442b-a256-73db9752e6ad None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:08.680851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 591/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:38:08 2022] PUT /identity/v3/projects/d1ddebe30dd94b87a5220ad6228fa27b/users/89014035004941d28d9dcc25e28979c0/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:38:10.675036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15d703-1f9f-438c-8218-5d9f2ba12158 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:10.675949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15d703-1f9f-438c-8218-5d9f2ba12158 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:10.675949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15d703-1f9f-438c-8218-5d9f2ba12158 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:10.678110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 602/1193] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:38:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:38:10.684453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c5e0c-e2d6-4b55-b327-cc83c14f76e1 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:10.685115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c5e0c-e2d6-4b55-b327-cc83c14f76e1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:10.685439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c5e0c-e2d6-4b55-b327-cc83c14f76e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:10.710060 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-161c5e0c-e2d6-4b55-b327-cc83c14f76e1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:38:10.711241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-161c5e0c-e2d6-4b55-b327-cc83c14f76e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:38:10.785458 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 592/1194] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:38:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:38:10.809134 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15d703-1f9f-438c-8218-5d9f2ba12158 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:10.857883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:10.925628 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15d703-1f9f-438c-8218-5d9f2ba12158 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:10.963965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436e8c7-5723-481c-bac5-bd45742e6f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:10.965174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436e8c7-5723-481c-bac5-bd45742e6f7d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:10.965580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436e8c7-5723-481c-bac5-bd45742e6f7d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:10.965972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436e8c7-5723-481c-bac5-bd45742e6f7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:10.978661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436e8c7-5723-481c-bac5-bd45742e6f7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-8436e8c7-5723-481c-bac5-bd45742e6f7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:38:10.986204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:38:10.987211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:38:10.988114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 603/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:38:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:38:10.993680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c5e0c-e2d6-4b55-b327-cc83c14f76e1 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:11.102373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c5e0c-e2d6-4b55-b327-cc83c14f76e1 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:11.112870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:11.141875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fef10-8a8f-43d0-a76c-df43281b0633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:11.142535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fef10-8a8f-43d0-a76c-df43281b0633 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:11.142811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fef10-8a8f-43d0-a76c-df43281b0633 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:11.143081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fef10-8a8f-43d0-a76c-df43281b0633 None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:11.143830 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fef10-8a8f-43d0-a76c-df43281b0633 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:38:11.144166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fef10-8a8f-43d0-a76c-df43281b0633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:11.145553 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fef10-8a8f-43d0-a76c-df43281b0633 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:11.153862 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 593/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:38:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:38:11.160024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436e8c7-5723-481c-bac5-bd45742e6f7d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:11.261633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:11.264781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436e8c7-5723-481c-bac5-bd45742e6f7d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:11.304542 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97118f6-e704-442c-a702-b418a45de953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:11.305838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97118f6-e704-442c-a702-b418a45de953 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:11.306290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97118f6-e704-442c-a702-b418a45de953 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:11.306734 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97118f6-e704-442c-a702-b418a45de953 None admin] PATH_INFO: `/v3/users/89014035004941d28d9dcc25e28979c0` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:11.308699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97118f6-e704-442c-a702-b418a45de953 None admin] RBAC: Authorizing `identity:get_user(user_id=89014035004941d28d9dcc25e28979c0)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:11.311010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97118f6-e704-442c-a702-b418a45de953 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:11.313652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 604/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Tue Jul 12 03:38:11 2022] GET /identity/v3/users/89014035004941d28d9dcc25e28979c0 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:38:11.319632 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fef10-8a8f-43d0-a76c-df43281b0633 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:11.433645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fef10-8a8f-43d0-a76c-df43281b0633 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:11.469364 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c3732-32b8-4160-9040-329cdeec39ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:11.469978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c3732-32b8-4160-9040-329cdeec39ec None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:11.470282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c3732-32b8-4160-9040-329cdeec39ec None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:11.470560 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c3732-32b8-4160-9040-329cdeec39ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:11.516510 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:11.547888 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c3732-32b8-4160-9040-329cdeec39ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:11.549497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c3732-32b8-4160-9040-329cdeec39ec None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:11.661446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 594/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:38:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:38:11.667479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97118f6-e704-442c-a702-b418a45de953 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:11.769321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97118f6-e704-442c-a702-b418a45de953 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:11.803773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acde167-5ab0-43f5-a525-d86224343f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:11.804687 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acde167-5ab0-43f5-a525-d86224343f20 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:11.804967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acde167-5ab0-43f5-a525-d86224343f20 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:11.805242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acde167-5ab0-43f5-a525-d86224343f20 None admin] PATH_INFO: `/v3/projects/d1ddebe30dd94b87a5220ad6228fa27b` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:11.806571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acde167-5ab0-43f5-a525-d86224343f20 None admin] RBAC: Authorizing `identity:get_project(project_id=d1ddebe30dd94b87a5220ad6228fa27b)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:11.808031 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acde167-5ab0-43f5-a525-d86224343f20 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:11.809731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 605/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Tue Jul 12 03:38:11 2022] GET /identity/v3/projects/d1ddebe30dd94b87a5220ad6228fa27b => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:38:11.816237 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c3732-32b8-4160-9040-329cdeec39ec None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:38:11.889558 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:11.920964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c3732-32b8-4160-9040-329cdeec39ec None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:38:11.954392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d79fbe-312e-4015-8a49-ecb83d06e720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:38:11.955054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d79fbe-312e-4015-8a49-ecb83d06e720 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:38:11.955319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d79fbe-312e-4015-8a49-ecb83d06e720 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:38:11.955590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d79fbe-312e-4015-8a49-ecb83d06e720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:38:11.956450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d79fbe-312e-4015-8a49-ecb83d06e720 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=d1ddebe30dd94b87a5220ad6228fa27b, user.id=89014035004941d28d9dcc25e28979c0) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:38:11.956751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d79fbe-312e-4015-8a49-ecb83d06e720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:38:11.958203 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d79fbe-312e-4015-8a49-ecb83d06e720 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:38:11.964115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 595/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:38:11 2022] GET /identity/v3/role_assignments?user.id=89014035004941d28d9dcc25e28979c0&scope.project.id=d1ddebe30dd94b87a5220ad6228fa27b&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:38:21.798711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=82 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:38:21.961556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=32 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:07.041236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829e454-3175-4e41-ac9e-65505a05971d None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:07.041851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829e454-3175-4e41-ac9e-65505a05971d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:07.041851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829e454-3175-4e41-ac9e-65505a05971d None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:07.043530 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 606/1201] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:39:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:39:07.053550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb09a35-8000-44b3-b060-6c0a612406aa None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:07.053550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb09a35-8000-44b3-b060-6c0a612406aa None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:07.053550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb09a35-8000-44b3-b060-6c0a612406aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:07.070840 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3cb09a35-8000-44b3-b060-6c0a612406aa None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:39:07.073638 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3cb09a35-8000-44b3-b060-6c0a612406aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:39:07.156718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 596/1202] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:39:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:39:07.187141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829e454-3175-4e41-ac9e-65505a05971d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:07.315818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829e454-3175-4e41-ac9e-65505a05971d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:07.367669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7aa97-2ffa-4c27-8868-e2a0493816c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:07.368672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7aa97-2ffa-4c27-8868-e2a0493816c8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:07.369250 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7aa97-2ffa-4c27-8868-e2a0493816c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:07.369630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7aa97-2ffa-4c27-8868-e2a0493816c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:07.384267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7aa97-2ffa-4c27-8868-e2a0493816c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:07.385752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7aa97-2ffa-4c27-8868-e2a0493816c8 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-aec7aa97-2ffa-4c27-8868-e2a0493816c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:07.413287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:07.414326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:07.415355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:07.415355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:07.415355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:07.415355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:07.417369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 607/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:39:07.424835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb09a35-8000-44b3-b060-6c0a612406aa None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:07.547811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb09a35-8000-44b3-b060-6c0a612406aa None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:07.584492 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:07.591972 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd73a34-9654-4311-ac6b-aed4b912a441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:07.592936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd73a34-9654-4311-ac6b-aed4b912a441 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:07.593418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd73a34-9654-4311-ac6b-aed4b912a441 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:07.594007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd73a34-9654-4311-ac6b-aed4b912a441 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:07.595185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd73a34-9654-4311-ac6b-aed4b912a441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:07.595641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd73a34-9654-4311-ac6b-aed4b912a441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:07.598283 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd73a34-9654-4311-ac6b-aed4b912a441 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:07.614693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 597/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:39:07.620660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7aa97-2ffa-4c27-8868-e2a0493816c8 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:07.693647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:07.756373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7aa97-2ffa-4c27-8868-e2a0493816c8 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:07.795190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83bc7f-5750-4fd2-bfa8-94a7cc8ce328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:07.795953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83bc7f-5750-4fd2-bfa8-94a7cc8ce328 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:07.796281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83bc7f-5750-4fd2-bfa8-94a7cc8ce328 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:07.796590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83bc7f-5750-4fd2-bfa8-94a7cc8ce328 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:07.797396 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83bc7f-5750-4fd2-bfa8-94a7cc8ce328 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:07.798988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83bc7f-5750-4fd2-bfa8-94a7cc8ce328 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:07.828069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 608/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Tue Jul 12 03:39:07 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:39:10.102733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a8f03-fd60-4b0b-957a-b2d1a784275a None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:10.103433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a8f03-fd60-4b0b-957a-b2d1a784275a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:10.103433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a8f03-fd60-4b0b-957a-b2d1a784275a None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:10.104719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 598/1206] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:39:10.111839 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1edfe3-193d-403e-83cd-72663b0b6efe None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:10.112376 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1edfe3-193d-403e-83cd-72663b0b6efe None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:10.112510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1edfe3-193d-403e-83cd-72663b0b6efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:10.132033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-4d1edfe3-193d-403e-83cd-72663b0b6efe None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:39:10.135419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4d1edfe3-193d-403e-83cd-72663b0b6efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:39:10.218777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 609/1207] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:39:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 03:39:10.245495 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a8f03-fd60-4b0b-957a-b2d1a784275a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:10.384744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a8f03-fd60-4b0b-957a-b2d1a784275a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:10.426522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439a9dc-7713-40aa-a806-0812774184b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:10.427360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439a9dc-7713-40aa-a806-0812774184b7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:10.427676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439a9dc-7713-40aa-a806-0812774184b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:10.427931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439a9dc-7713-40aa-a806-0812774184b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:10.440986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6439a9dc-7713-40aa-a806-0812774184b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6439a9dc-7713-40aa-a806-0812774184b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:39:10.452158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:10.453123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:10.462313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 599/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:39:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:39:10.463076 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1edfe3-193d-403e-83cd-72663b0b6efe None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:10.511200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:10.591111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1edfe3-193d-403e-83cd-72663b0b6efe None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:10.629597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2532d-4212-46c0-a4fc-0777a4a1ea66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:10.630993 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2532d-4212-46c0-a4fc-0777a4a1ea66 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:10.631896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2532d-4212-46c0-a4fc-0777a4a1ea66 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:10.632212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2532d-4212-46c0-a4fc-0777a4a1ea66 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:10.633021 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2532d-4212-46c0-a4fc-0777a4a1ea66 None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:10.633328 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2532d-4212-46c0-a4fc-0777a4a1ea66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:10.634820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2532d-4212-46c0-a4fc-0777a4a1ea66 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:10.644865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 610/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:39:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:10.651967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439a9dc-7713-40aa-a806-0812774184b7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:10.711927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:10.775340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439a9dc-7713-40aa-a806-0812774184b7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:10.822346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4608-981b-41e4-91c9-e41a7244fa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:10.823156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4608-981b-41e4-91c9-e41a7244fa21 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:10.823444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4608-981b-41e4-91c9-e41a7244fa21 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:10.823713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4608-981b-41e4-91c9-e41a7244fa21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:10.838261 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b4608-981b-41e4-91c9-e41a7244fa21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:10.840073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044b4608-981b-41e4-91c9-e41a7244fa21 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-044b4608-981b-41e4-91c9-e41a7244fa21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:10.862086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:10.863064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:10.863980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:10.863980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:10.863980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:10.863980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:10.863980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 600/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:10.869893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2532d-4212-46c0-a4fc-0777a4a1ea66 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:10.994171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:10.997603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2532d-4212-46c0-a4fc-0777a4a1ea66 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:11.042983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213d7cc-33d8-48dd-9722-f90a90ce5cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:11.044328 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213d7cc-33d8-48dd-9722-f90a90ce5cb5 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:11.044727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213d7cc-33d8-48dd-9722-f90a90ce5cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:11.045184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213d7cc-33d8-48dd-9722-f90a90ce5cb5 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:11.046244 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213d7cc-33d8-48dd-9722-f90a90ce5cb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:11.046733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213d7cc-33d8-48dd-9722-f90a90ce5cb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:11.048187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213d7cc-33d8-48dd-9722-f90a90ce5cb5 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:11.062253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 611/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:11.068360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4608-981b-41e4-91c9-e41a7244fa21 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:11.189504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4608-981b-41e4-91c9-e41a7244fa21 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:11.191348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:11.230944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8ae65-8d57-4138-9727-4d77ffb88b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:11.231824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8ae65-8d57-4138-9727-4d77ffb88b69 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:11.232154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8ae65-8d57-4138-9727-4d77ffb88b69 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:11.232434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8ae65-8d57-4138-9727-4d77ffb88b69 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:11.245445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8ae65-8d57-4138-9727-4d77ffb88b69 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:11.247446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b8ae65-8d57-4138-9727-4d77ffb88b69 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-b0b8ae65-8d57-4138-9727-4d77ffb88b69 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:39:11.263849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:11.265474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:11.266544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:11.266544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:11.266544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:39:11.266544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:39:11.266544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:39:11.266544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:39:11.266544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:39:11.266544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:39:11.266544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 03:39:11.266544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:11.266544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 601/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:11.271432 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213d7cc-33d8-48dd-9722-f90a90ce5cb5 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:11.399215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213d7cc-33d8-48dd-9722-f90a90ce5cb5 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:11.449034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1d09b-6ce1-4ffb-88d0-97ca83e21fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:11.449798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1d09b-6ce1-4ffb-88d0-97ca83e21fc2 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:11.450087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1d09b-6ce1-4ffb-88d0-97ca83e21fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:11.450369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1d09b-6ce1-4ffb-88d0-97ca83e21fc2 None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:11.451385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1d09b-6ce1-4ffb-88d0-97ca83e21fc2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:11.452820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1d09b-6ce1-4ffb-88d0-97ca83e21fc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:11.454961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1d09b-6ce1-4ffb-88d0-97ca83e21fc2 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:11.484032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 612/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Jul 12 03:39:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:11.486578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8ae65-8d57-4138-9727-4d77ffb88b69 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:11.606265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8ae65-8d57-4138-9727-4d77ffb88b69 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:11.660975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68311247-3fd8-4cc9-8ade-6d4c315c1ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:11.661688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68311247-3fd8-4cc9-8ade-6d4c315c1ffe None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:11.662087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68311247-3fd8-4cc9-8ade-6d4c315c1ffe None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:11.662445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68311247-3fd8-4cc9-8ade-6d4c315c1ffe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:11.703056 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:11.756442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68311247-3fd8-4cc9-8ade-6d4c315c1ffe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:11.758041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68311247-3fd8-4cc9-8ade-6d4c315c1ffe None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:11.875729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 602/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:39:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:39:11.881682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1d09b-6ce1-4ffb-88d0-97ca83e21fc2 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:11.902447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:11.999445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1d09b-6ce1-4ffb-88d0-97ca83e21fc2 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:12.034087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3d9d1-fe28-428f-a957-f53233616525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:12.034878 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3d9d1-fe28-428f-a957-f53233616525 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:12.035161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3d9d1-fe28-428f-a957-f53233616525 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:12.035461 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3d9d1-fe28-428f-a957-f53233616525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:12.045934 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3d9d1-fe28-428f-a957-f53233616525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:12.047397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3d9d1-fe28-428f-a957-f53233616525 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-c1f3d9d1-fe28-428f-a957-f53233616525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:12.065069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:12.065989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:12.066927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:12.066927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:12.066927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:12.066927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:12.066927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 613/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:39:12.072128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68311247-3fd8-4cc9-8ade-6d4c315c1ffe None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:12.143333 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:12.183509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68311247-3fd8-4cc9-8ade-6d4c315c1ffe None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:12.224270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6af84b-388d-4094-a5ed-6f374b4fec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:12.224983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6af84b-388d-4094-a5ed-6f374b4fec65 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:12.225239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6af84b-388d-4094-a5ed-6f374b4fec65 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:12.225507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6af84b-388d-4094-a5ed-6f374b4fec65 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:12.226247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6af84b-388d-4094-a5ed-6f374b4fec65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:12.226536 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6af84b-388d-4094-a5ed-6f374b4fec65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:12.227980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6af84b-388d-4094-a5ed-6f374b4fec65 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:12.241182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 603/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:39:12.246531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3d9d1-fe28-428f-a957-f53233616525 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:12.321042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:12.359379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3d9d1-fe28-428f-a957-f53233616525 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:12.397812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b093d3-e3fa-45c7-8fa2-9013ed0e8077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:12.398606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b093d3-e3fa-45c7-8fa2-9013ed0e8077 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:12.398892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b093d3-e3fa-45c7-8fa2-9013ed0e8077 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:12.399168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b093d3-e3fa-45c7-8fa2-9013ed0e8077 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:12.407030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b093d3-e3fa-45c7-8fa2-9013ed0e8077 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:12.408572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b093d3-e3fa-45c7-8fa2-9013ed0e8077 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-d7b093d3-e3fa-45c7-8fa2-9013ed0e8077 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:39:12.426081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:12.427064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:12.428378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:39:12.428378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:39:12.428378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:39:12.428378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:39:12.428378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:39:12.428378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:39:12.428378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:39:12.428378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:12.428378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 614/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:39:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:12.433288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6af84b-388d-4094-a5ed-6f374b4fec65 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:12.541060 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6af84b-388d-4094-a5ed-6f374b4fec65 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:12.567709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:12.582860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6bc32-6be0-4acc-b869-af8436c763bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:12.583530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6bc32-6be0-4acc-b869-af8436c763bf None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:12.583812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6bc32-6be0-4acc-b869-af8436c763bf None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:12.584126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6bc32-6be0-4acc-b869-af8436c763bf None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:12.584925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6bc32-6be0-4acc-b869-af8436c763bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:12.585232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6bc32-6be0-4acc-b869-af8436c763bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:12.586671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6bc32-6be0-4acc-b869-af8436c763bf None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:12.603541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 604/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:39:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:12.610613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b093d3-e3fa-45c7-8fa2-9013ed0e8077 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:12.730218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b093d3-e3fa-45c7-8fa2-9013ed0e8077 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:12.768754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:12.775769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01314130-1782-4f5f-a3da-05fdb80841ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:12.776506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01314130-1782-4f5f-a3da-05fdb80841ca None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:12.776822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01314130-1782-4f5f-a3da-05fdb80841ca None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:12.777370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01314130-1782-4f5f-a3da-05fdb80841ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:12.777903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01314130-1782-4f5f-a3da-05fdb80841ca None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=eded06390a2744c9aa52f77e3bfc2db9) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:12.778224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01314130-1782-4f5f-a3da-05fdb80841ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:12.779705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01314130-1782-4f5f-a3da-05fdb80841ca None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:12.787568 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 615/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:39:12 2022] GET /identity/v3/role_assignments?user.id=eded06390a2744c9aa52f77e3bfc2db9&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:15.043777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac235-2c83-47f4-9a22-4cc1a34c0259 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:15.044361 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac235-2c83-47f4-9a22-4cc1a34c0259 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:15.044361 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ac235-2c83-47f4-9a22-4cc1a34c0259 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:15.045717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 605/1220] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:39:15.051503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb591ab-03ef-4153-a018-5546c566f832 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:15.051933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb591ab-03ef-4153-a018-5546c566f832 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:15.052073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb591ab-03ef-4153-a018-5546c566f832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:15.070711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-1eb591ab-03ef-4153-a018-5546c566f832 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:39:15.071937 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1eb591ab-03ef-4153-a018-5546c566f832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:39:15.148715 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 616/1221] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:39:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:39:15.171015 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac235-2c83-47f4-9a22-4cc1a34c0259 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:15.283233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ac235-2c83-47f4-9a22-4cc1a34c0259 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:15.321529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01332c54-4bc6-4af7-a34b-b9a9884dcd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:15.322394 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01332c54-4bc6-4af7-a34b-b9a9884dcd62 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:15.322678 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01332c54-4bc6-4af7-a34b-b9a9884dcd62 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:15.322952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01332c54-4bc6-4af7-a34b-b9a9884dcd62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:15.333150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01332c54-4bc6-4af7-a34b-b9a9884dcd62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-01332c54-4bc6-4af7-a34b-b9a9884dcd62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:39:15.340868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:15.342250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:15.343291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 606/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:39:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:15.348309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb591ab-03ef-4153-a018-5546c566f832 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:15.451709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb591ab-03ef-4153-a018-5546c566f832 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:15.480322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:15.491901 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17e3e4-e90d-41b9-8426-ac06e4d29677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:15.492576 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17e3e4-e90d-41b9-8426-ac06e4d29677 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:15.492856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17e3e4-e90d-41b9-8426-ac06e4d29677 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:15.493126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17e3e4-e90d-41b9-8426-ac06e4d29677 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:15.493868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17e3e4-e90d-41b9-8426-ac06e4d29677 None admin] RBAC: Adding query filter params (name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:15.494170 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17e3e4-e90d-41b9-8426-ac06e4d29677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:15.495541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17e3e4-e90d-41b9-8426-ac06e4d29677 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:15.503906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 617/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:39:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:15.510084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01332c54-4bc6-4af7-a34b-b9a9884dcd62 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:15.557836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:15.613647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01332c54-4bc6-4af7-a34b-b9a9884dcd62 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:15.650409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdc971-2010-42b5-88f5-3e6b37cb2701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:15.651210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdc971-2010-42b5-88f5-3e6b37cb2701 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:15.651537 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdc971-2010-42b5-88f5-3e6b37cb2701 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:15.651803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdc971-2010-42b5-88f5-3e6b37cb2701 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:15.666990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdc971-2010-42b5-88f5-3e6b37cb2701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:15.668962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdc971-2010-42b5-88f5-3e6b37cb2701 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6bfdc971-2010-42b5-88f5-3e6b37cb2701 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:15.692123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:15.693287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:15.699153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:15.699153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:15.699153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:15.699153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:15.699153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 607/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:15.699694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17e3e4-e90d-41b9-8426-ac06e4d29677 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:15.807155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17e3e4-e90d-41b9-8426-ac06e4d29677 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:15.840716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96d58b-eb82-48d8-ab34-c24a731415c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:15.841299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96d58b-eb82-48d8-ab34-c24a731415c9 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:15.841582 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96d58b-eb82-48d8-ab34-c24a731415c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:15.841854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96d58b-eb82-48d8-ab34-c24a731415c9 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:15.842563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96d58b-eb82-48d8-ab34-c24a731415c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:15.842863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96d58b-eb82-48d8-ab34-c24a731415c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:15.844251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96d58b-eb82-48d8-ab34-c24a731415c9 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:15.862794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 618/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:15.864284 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdc971-2010-42b5-88f5-3e6b37cb2701 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:15.971791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:15.979174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdc971-2010-42b5-88f5-3e6b37cb2701 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:16.017319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5997c35-b6b5-4a97-b277-c46330942ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:16.018225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5997c35-b6b5-4a97-b277-c46330942ebe None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:16.018552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5997c35-b6b5-4a97-b277-c46330942ebe None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:16.018883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5997c35-b6b5-4a97-b277-c46330942ebe None admin] PATH_INFO: `/v3/users/placement` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:16.029014 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5997c35-b6b5-4a97-b277-c46330942ebe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:16.030893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5997c35-b6b5-4a97-b277-c46330942ebe None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-c5997c35-b6b5-4a97-b277-c46330942ebe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:39:16.048176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:16.049709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:16.050662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:16.050662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:16.050662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:39:16.050662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:39:16.050662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:39:16.050662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:39:16.050662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:39:16.050662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:39:16.050662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 03:39:16.050662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:16.050662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 608/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:16.055477 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96d58b-eb82-48d8-ab34-c24a731415c9 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:16.106487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:16.165487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96d58b-eb82-48d8-ab34-c24a731415c9 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:16.204732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c88b1d-cd44-4654-8307-605a36f82aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:16.205364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c88b1d-cd44-4654-8307-605a36f82aed None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:16.205738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c88b1d-cd44-4654-8307-605a36f82aed None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:16.205946 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c88b1d-cd44-4654-8307-605a36f82aed None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:16.206817 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c88b1d-cd44-4654-8307-605a36f82aed None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:16.207110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c88b1d-cd44-4654-8307-605a36f82aed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:16.208683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c88b1d-cd44-4654-8307-605a36f82aed None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:16.226822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 619/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Jul 12 03:39:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:16.232746 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5997c35-b6b5-4a97-b277-c46330942ebe None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:16.344057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5997c35-b6b5-4a97-b277-c46330942ebe None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:16.384065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70fa39-edd3-4db2-9e84-5cb04999a6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:16.384807 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70fa39-edd3-4db2-9e84-5cb04999a6bc None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:16.385090 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70fa39-edd3-4db2-9e84-5cb04999a6bc None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:16.385371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70fa39-edd3-4db2-9e84-5cb04999a6bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:16.396917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70fa39-edd3-4db2-9e84-5cb04999a6bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:16.398805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70fa39-edd3-4db2-9e84-5cb04999a6bc None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-6d70fa39-edd3-4db2-9e84-5cb04999a6bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:16.422648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:16.423995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:16.424923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:16.424923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:16.424923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:16.424923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:16.424923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 609/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:16.431260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c88b1d-cd44-4654-8307-605a36f82aed None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:16.547431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c88b1d-cd44-4654-8307-605a36f82aed None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:16.549835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:16.589055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb0182-3d1b-43d6-afb2-10f02223d2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:16.589702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb0182-3d1b-43d6-afb2-10f02223d2e1 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:16.589988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb0182-3d1b-43d6-afb2-10f02223d2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:16.590287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb0182-3d1b-43d6-afb2-10f02223d2e1 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:16.590987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb0182-3d1b-43d6-afb2-10f02223d2e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:16.591291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb0182-3d1b-43d6-afb2-10f02223d2e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:16.592710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb0182-3d1b-43d6-afb2-10f02223d2e1 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:16.604784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 620/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:16.612115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70fa39-edd3-4db2-9e84-5cb04999a6bc None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:16.621810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:16.736431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70fa39-edd3-4db2-9e84-5cb04999a6bc None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:16.779229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c028766-fa1f-4cf8-b291-65189cf3c6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:16.780287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c028766-fa1f-4cf8-b291-65189cf3c6ae None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:16.780639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c028766-fa1f-4cf8-b291-65189cf3c6ae None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:16.780970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c028766-fa1f-4cf8-b291-65189cf3c6ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:16.789301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c028766-fa1f-4cf8-b291-65189cf3c6ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:16.791155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c028766-fa1f-4cf8-b291-65189cf3c6ae None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-1c028766-fa1f-4cf8-b291-65189cf3c6ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:39:16.807375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:16.808493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:16.809827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:39:16.809827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:39:16.809827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:39:16.809827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:39:16.809827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:39:16.809827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:39:16.809827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:39:16.809827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:16.809827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 610/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:39:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:16.815676 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb0182-3d1b-43d6-afb2-10f02223d2e1 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:16.932057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb0182-3d1b-43d6-afb2-10f02223d2e1 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:16.973345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6a950-ccdc-4cf5-9053-4a4521524beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:16.974118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6a950-ccdc-4cf5-9053-4a4521524beb None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:16.974475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6a950-ccdc-4cf5-9053-4a4521524beb None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:16.974892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6a950-ccdc-4cf5-9053-4a4521524beb None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:16.975796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6a950-ccdc-4cf5-9053-4a4521524beb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:16.976229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6a950-ccdc-4cf5-9053-4a4521524beb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:16.977932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6a950-ccdc-4cf5-9053-4a4521524beb None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:16.998154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 621/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:39:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:16.999428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c028766-fa1f-4cf8-b291-65189cf3c6ae None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:17.069425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:17.107411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c028766-fa1f-4cf8-b291-65189cf3c6ae None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:17.145183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd224a-2018-4fcf-949e-b2d98d67fedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:17.146164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd224a-2018-4fcf-949e-b2d98d67fedc None admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:17.146459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd224a-2018-4fcf-949e-b2d98d67fedc None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:17.146769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd224a-2018-4fcf-949e-b2d98d67fedc None admin] PATH_INFO: `/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/eded06390a2744c9aa52f77e3bfc2db9/roles/f653b35aa347415e953be7ead75d9fd0` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:17.163296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd224a-2018-4fcf-949e-b2d98d67fedc None admin] RBAC: Authorizing `identity:create_grant(project_id=48b17053daa84e4fa94840777cb1bee6, user_id=eded06390a2744c9aa52f77e3bfc2db9, role_id=f653b35aa347415e953be7ead75d9fd0)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:17.165003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd224a-2018-4fcf-949e-b2d98d67fedc None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:17.179021 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 611/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:39:16 2022] PUT /identity/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/eded06390a2744c9aa52f77e3bfc2db9/roles/f653b35aa347415e953be7ead75d9fd0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:39:19.191109 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad391c4-6f01-4cf2-b131-1191662cb6ea None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:19.191656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad391c4-6f01-4cf2-b131-1191662cb6ea None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:19.191656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad391c4-6f01-4cf2-b131-1191662cb6ea None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:19.193057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 622/1233] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:39:19.198463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f6476-750c-459b-8ee2-b5a9e4155a4b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:19.199087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f6476-750c-459b-8ee2-b5a9e4155a4b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:19.199087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f6476-750c-459b-8ee2-b5a9e4155a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:19.217584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-665f6476-750c-459b-8ee2-b5a9e4155a4b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:39:19.218724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-665f6476-750c-459b-8ee2-b5a9e4155a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:39:19.292442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 612/1234] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:39:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:39:19.322295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad391c4-6f01-4cf2-b131-1191662cb6ea None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:19.356131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:19.441624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad391c4-6f01-4cf2-b131-1191662cb6ea None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:19.478783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f34399-d194-48f6-aac3-b46e1fccce83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:19.479595 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f34399-d194-48f6-aac3-b46e1fccce83 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:19.479948 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f34399-d194-48f6-aac3-b46e1fccce83 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:19.480264 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f34399-d194-48f6-aac3-b46e1fccce83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:19.492459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f34399-d194-48f6-aac3-b46e1fccce83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-c5f34399-d194-48f6-aac3-b46e1fccce83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:39:19.500260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:19.501438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:19.502312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 623/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Tue Jul 12 03:39:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:19.509320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f6476-750c-459b-8ee2-b5a9e4155a4b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:19.563863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:19.630759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f6476-750c-459b-8ee2-b5a9e4155a4b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:19.681434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ffbd6-3f13-4ca1-baa6-bf97c54750ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:19.682476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ffbd6-3f13-4ca1-baa6-bf97c54750ee None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:19.683163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ffbd6-3f13-4ca1-baa6-bf97c54750ee None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:19.683732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ffbd6-3f13-4ca1-baa6-bf97c54750ee None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:19.684756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ffbd6-3f13-4ca1-baa6-bf97c54750ee None admin] RBAC: Adding query filter params (name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:19.685204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ffbd6-3f13-4ca1-baa6-bf97c54750ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:19.687912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ffbd6-3f13-4ca1-baa6-bf97c54750ee None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:19.700654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 613/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Tue Jul 12 03:39:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:19.708364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f34399-d194-48f6-aac3-b46e1fccce83 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:19.801018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:19.818251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f34399-d194-48f6-aac3-b46e1fccce83 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:19.857625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab556f92-2ae2-45b6-a50a-49f87456345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:19.858691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab556f92-2ae2-45b6-a50a-49f87456345b None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:19.859055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab556f92-2ae2-45b6-a50a-49f87456345b None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:19.859339 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab556f92-2ae2-45b6-a50a-49f87456345b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:19.874736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab556f92-2ae2-45b6-a50a-49f87456345b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:19.876808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab556f92-2ae2-45b6-a50a-49f87456345b None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-ab556f92-2ae2-45b6-a50a-49f87456345b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:19.899589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:19.901006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:19.901996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:19.901996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:19.901996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:19.901996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:19.901996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 624/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:19.910365 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ffbd6-3f13-4ca1-baa6-bf97c54750ee None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:20.024152 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ffbd6-3f13-4ca1-baa6-bf97c54750ee None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:20.035011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:20.070586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef10817-23b0-49d9-8f6a-1260b1dbc21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:20.071567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef10817-23b0-49d9-8f6a-1260b1dbc21c None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:20.071896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef10817-23b0-49d9-8f6a-1260b1dbc21c None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:20.072415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef10817-23b0-49d9-8f6a-1260b1dbc21c None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:20.073906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef10817-23b0-49d9-8f6a-1260b1dbc21c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:20.074326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef10817-23b0-49d9-8f6a-1260b1dbc21c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:20.076687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef10817-23b0-49d9-8f6a-1260b1dbc21c None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:20.091432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 614/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:20.097477 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab556f92-2ae2-45b6-a50a-49f87456345b None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:20.205054 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab556f92-2ae2-45b6-a50a-49f87456345b None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:20.242967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a7444-0d58-465b-b0e9-6126a11da72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:20.243749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a7444-0d58-465b-b0e9-6126a11da72a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:20.244153 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a7444-0d58-465b-b0e9-6126a11da72a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:20.244437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a7444-0d58-465b-b0e9-6126a11da72a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:20.248409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:20.255130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a7444-0d58-465b-b0e9-6126a11da72a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:20.256763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a7444-0d58-465b-b0e9-6126a11da72a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-be1a7444-0d58-465b-b0e9-6126a11da72a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:39:20.273215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:20.274399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:20.275681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:20.275681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:20.275681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:39:20.275681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:39:20.275681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:39:20.275681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:39:20.275681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:39:20.275681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:39:20.275681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 03:39:20.275681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:20.275681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 625/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:20.281399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef10817-23b0-49d9-8f6a-1260b1dbc21c None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:20.404100 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef10817-23b0-49d9-8f6a-1260b1dbc21c None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:20.440703 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d6c67-e479-45b5-8af6-68d312e31c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:20.441450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d6c67-e479-45b5-8af6-68d312e31c79 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:20.441752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d6c67-e479-45b5-8af6-68d312e31c79 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:20.442032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d6c67-e479-45b5-8af6-68d312e31c79 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:20.442922 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d6c67-e479-45b5-8af6-68d312e31c79 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:20.443253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d6c67-e479-45b5-8af6-68d312e31c79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:20.444775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d6c67-e479-45b5-8af6-68d312e31c79 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:20.464350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 615/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Jul 12 03:39:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:20.470335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a7444-0d58-465b-b0e9-6126a11da72a None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:20.585153 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a7444-0d58-465b-b0e9-6126a11da72a None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:20.621486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab03931-1371-4e16-84c6-3e5e7495d31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:20.622275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab03931-1371-4e16-84c6-3e5e7495d31e None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:20.622648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab03931-1371-4e16-84c6-3e5e7495d31e None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:20.623033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab03931-1371-4e16-84c6-3e5e7495d31e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:20.702573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:20.710986 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab03931-1371-4e16-84c6-3e5e7495d31e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:20.712566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab03931-1371-4e16-84c6-3e5e7495d31e None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:20.822551 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 626/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:39:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:39:20.823881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d6c67-e479-45b5-8af6-68d312e31c79 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:20.854199 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:20.930367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d6c67-e479-45b5-8af6-68d312e31c79 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:20.971776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2d4f8-2f4e-4998-af6d-02e57edfebb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:20.972662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2d4f8-2f4e-4998-af6d-02e57edfebb6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:20.972927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2d4f8-2f4e-4998-af6d-02e57edfebb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:20.973202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2d4f8-2f4e-4998-af6d-02e57edfebb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:20.983607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2d4f8-2f4e-4998-af6d-02e57edfebb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:20.985156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2d4f8-2f4e-4998-af6d-02e57edfebb6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-74d2d4f8-2f4e-4998-af6d-02e57edfebb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:21.002904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:21.004161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:21.005278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:21.005278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:21.005278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:21.005278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:21.005278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 616/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:21.011131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab03931-1371-4e16-84c6-3e5e7495d31e None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:21.118883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:21.125339 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab03931-1371-4e16-84c6-3e5e7495d31e None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:21.161814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29401b-990d-41cd-8e2a-a8c41598804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:21.162716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29401b-990d-41cd-8e2a-a8c41598804d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:21.163092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29401b-990d-41cd-8e2a-a8c41598804d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:21.163494 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29401b-990d-41cd-8e2a-a8c41598804d None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:21.164400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29401b-990d-41cd-8e2a-a8c41598804d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:21.164832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29401b-990d-41cd-8e2a-a8c41598804d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:21.166481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29401b-990d-41cd-8e2a-a8c41598804d None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:21.177991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 627/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:21.183922 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2d4f8-2f4e-4998-af6d-02e57edfebb6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:21.271124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:21.293698 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2d4f8-2f4e-4998-af6d-02e57edfebb6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:21.329016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01eb82-8295-4245-b8e6-88fcd37b6a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:21.329816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01eb82-8295-4245-b8e6-88fcd37b6a0f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:21.330100 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01eb82-8295-4245-b8e6-88fcd37b6a0f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:21.330371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01eb82-8295-4245-b8e6-88fcd37b6a0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:21.338798 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01eb82-8295-4245-b8e6-88fcd37b6a0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:21.340327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01eb82-8295-4245-b8e6-88fcd37b6a0f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-cc01eb82-8295-4245-b8e6-88fcd37b6a0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:39:21.357769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:21.358794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:21.359676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:39:21.359676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:39:21.359676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:39:21.359676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:39:21.359676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:39:21.359676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:39:21.359676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:39:21.359676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:21.359676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 617/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:39:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:21.366020 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29401b-990d-41cd-8e2a-a8c41598804d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:21.477401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29401b-990d-41cd-8e2a-a8c41598804d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:21.516087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76206c9d-ace9-4d66-8977-614312b71457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:21.516910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76206c9d-ace9-4d66-8977-614312b71457 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:21.517253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76206c9d-ace9-4d66-8977-614312b71457 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:21.517695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76206c9d-ace9-4d66-8977-614312b71457 None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:21.518670 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76206c9d-ace9-4d66-8977-614312b71457 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:21.519096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76206c9d-ace9-4d66-8977-614312b71457 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:21.520983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76206c9d-ace9-4d66-8977-614312b71457 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:21.535603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 628/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:39:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:21.541781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01eb82-8295-4245-b8e6-88fcd37b6a0f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:21.666430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01eb82-8295-4245-b8e6-88fcd37b6a0f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:21.713380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d54b3ce-390a-4a30-8efe-e3ce1870f8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:21.714047 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54b3ce-390a-4a30-8efe-e3ce1870f8eb None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:21.714335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54b3ce-390a-4a30-8efe-e3ce1870f8eb None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:21.714616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54b3ce-390a-4a30-8efe-e3ce1870f8eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:21.715455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54b3ce-390a-4a30-8efe-e3ce1870f8eb None admin] RBAC: Adding query filter params (role.id=f653b35aa347415e953be7ead75d9fd0, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=eded06390a2744c9aa52f77e3bfc2db9) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:21.715770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54b3ce-390a-4a30-8efe-e3ce1870f8eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:21.717279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d54b3ce-390a-4a30-8efe-e3ce1870f8eb None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:21.724309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 618/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:39:21 2022] GET /identity/v3/role_assignments?user.id=eded06390a2744c9aa52f77e3bfc2db9&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=f653b35aa347415e953be7ead75d9fd0 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:23.857172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a2a98-0696-4561-8bea-c9ca02310e87 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:23.857791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a2a98-0696-4561-8bea-c9ca02310e87 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:23.857791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a2a98-0696-4561-8bea-c9ca02310e87 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:23.859238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 629/1247] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:39:23.864251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021acca4-5e61-470a-b15e-2867452bf544 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:23.864768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021acca4-5e61-470a-b15e-2867452bf544 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:23.864863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021acca4-5e61-470a-b15e-2867452bf544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:23.870153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:23.884619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-021acca4-5e61-470a-b15e-2867452bf544 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:39:23.885931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-021acca4-5e61-470a-b15e-2867452bf544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:39:23.957606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 619/1248] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:39:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:39:23.981380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a2a98-0696-4561-8bea-c9ca02310e87 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:24.002256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:24.095728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a2a98-0696-4561-8bea-c9ca02310e87 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:24.134758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c230f1-e770-4948-9aaa-b8abd22eea74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:24.135621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c230f1-e770-4948-9aaa-b8abd22eea74 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:24.135961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c230f1-e770-4948-9aaa-b8abd22eea74 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:24.136450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c230f1-e770-4948-9aaa-b8abd22eea74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:24.152299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c230f1-e770-4948-9aaa-b8abd22eea74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-e3c230f1-e770-4948-9aaa-b8abd22eea74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:39:24.161832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:24.163005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:24.163867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 630/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:39:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:24.170156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021acca4-5e61-470a-b15e-2867452bf544 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:24.285836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021acca4-5e61-470a-b15e-2867452bf544 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:24.322661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:24.328304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16f991-b7eb-4fe9-9b5f-fa401958353f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:24.328989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16f991-b7eb-4fe9-9b5f-fa401958353f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:24.329286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16f991-b7eb-4fe9-9b5f-fa401958353f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:24.329763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16f991-b7eb-4fe9-9b5f-fa401958353f None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:24.330581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16f991-b7eb-4fe9-9b5f-fa401958353f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:24.331022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16f991-b7eb-4fe9-9b5f-fa401958353f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:24.332820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16f991-b7eb-4fe9-9b5f-fa401958353f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:24.341990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 620/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:39:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:39:24.348513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c230f1-e770-4948-9aaa-b8abd22eea74 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:24.433844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:24.461358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c230f1-e770-4948-9aaa-b8abd22eea74 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:24.500837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36a5d8-a725-4572-94df-745a88e715a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:24.501647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36a5d8-a725-4572-94df-745a88e715a6 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:24.501924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36a5d8-a725-4572-94df-745a88e715a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:24.502206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36a5d8-a725-4572-94df-745a88e715a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:24.513533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36a5d8-a725-4572-94df-745a88e715a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:24.515163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36a5d8-a725-4572-94df-745a88e715a6 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-9a36a5d8-a725-4572-94df-745a88e715a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:24.535821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:24.537102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:24.538442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:24.538442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:24.538442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:24.538442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:24.538442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 631/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:39:24.544466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16f991-b7eb-4fe9-9b5f-fa401958353f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:24.667371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16f991-b7eb-4fe9-9b5f-fa401958353f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:24.713215 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fb488-425c-4651-a524-616301e13a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:24.713927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fb488-425c-4651-a524-616301e13a79 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:24.714291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fb488-425c-4651-a524-616301e13a79 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:24.714618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fb488-425c-4651-a524-616301e13a79 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:24.715413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fb488-425c-4651-a524-616301e13a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:24.715806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fb488-425c-4651-a524-616301e13a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:24.717395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fb488-425c-4651-a524-616301e13a79 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:24.737684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 621/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:24.738968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36a5d8-a725-4572-94df-745a88e715a6 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:24.854258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36a5d8-a725-4572-94df-745a88e715a6 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:24.883340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:24.892506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1754e-5190-4b9a-8599-63f4013f2887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:24.893744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1754e-5190-4b9a-8599-63f4013f2887 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:24.894286 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1754e-5190-4b9a-8599-63f4013f2887 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:24.894621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1754e-5190-4b9a-8599-63f4013f2887 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:24.906133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1754e-5190-4b9a-8599-63f4013f2887 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:24.908279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1754e-5190-4b9a-8599-63f4013f2887 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-e3c1754e-5190-4b9a-8599-63f4013f2887 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:39:24.927145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:24.928172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:24.929374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:24.929374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:24.929374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:39:24.929374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:39:24.929374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:39:24.929374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:39:24.929374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:39:24.929374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:39:24.929374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 03:39:24.929374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:24.929374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 632/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:24.934513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fb488-425c-4651-a524-616301e13a79 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:25.001166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:25.056833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fb488-425c-4651-a524-616301e13a79 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:25.100213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4095b3-61cb-437a-a547-f90853d86ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:25.100886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4095b3-61cb-437a-a547-f90853d86ef7 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:25.101163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4095b3-61cb-437a-a547-f90853d86ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:25.101432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4095b3-61cb-437a-a547-f90853d86ef7 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:25.102264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4095b3-61cb-437a-a547-f90853d86ef7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:25.102617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4095b3-61cb-437a-a547-f90853d86ef7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:25.104289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4095b3-61cb-437a-a547-f90853d86ef7 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:25.123467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 622/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Jul 12 03:39:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:25.129775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1754e-5190-4b9a-8599-63f4013f2887 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:25.239007 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1754e-5190-4b9a-8599-63f4013f2887 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:25.285923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4b239-ec52-4c5f-b132-5d2ddcf9d493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:25.286644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4b239-ec52-4c5f-b132-5d2ddcf9d493 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:25.286932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4b239-ec52-4c5f-b132-5d2ddcf9d493 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:25.287213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4b239-ec52-4c5f-b132-5d2ddcf9d493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:25.352986 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:25.370893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4b239-ec52-4c5f-b132-5d2ddcf9d493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:25.373492 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c4b239-ec52-4c5f-b132-5d2ddcf9d493 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:25.483946 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 633/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:39:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:39:25.491496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4095b3-61cb-437a-a547-f90853d86ef7 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:25.615650 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4095b3-61cb-437a-a547-f90853d86ef7 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:25.629158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:25.653557 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c44c3-c9c3-4d25-9e95-1c71bf7c46f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:25.654535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c44c3-c9c3-4d25-9e95-1c71bf7c46f1 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:25.654840 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c44c3-c9c3-4d25-9e95-1c71bf7c46f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:25.655128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c44c3-c9c3-4d25-9e95-1c71bf7c46f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:25.669176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c44c3-c9c3-4d25-9e95-1c71bf7c46f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:25.671054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c44c3-c9c3-4d25-9e95-1c71bf7c46f1 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-e68c44c3-c9c3-4d25-9e95-1c71bf7c46f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:25.694435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:25.695655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:25.696800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:25.696800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:25.696800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:25.696800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:25.696800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 623/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:25.701960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4b239-ec52-4c5f-b132-5d2ddcf9d493 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:25.801458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:25.814885 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4b239-ec52-4c5f-b132-5d2ddcf9d493 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:25.855788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68a9db-3f1b-4cc6-8bb5-1bcf95aa9677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:25.856545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68a9db-3f1b-4cc6-8bb5-1bcf95aa9677 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:25.856835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68a9db-3f1b-4cc6-8bb5-1bcf95aa9677 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:25.857140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68a9db-3f1b-4cc6-8bb5-1bcf95aa9677 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:25.858036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68a9db-3f1b-4cc6-8bb5-1bcf95aa9677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:25.858402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68a9db-3f1b-4cc6-8bb5-1bcf95aa9677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:25.860180 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68a9db-3f1b-4cc6-8bb5-1bcf95aa9677 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:25.875586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 634/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:39:25.881871 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c44c3-c9c3-4d25-9e95-1c71bf7c46f1 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:25.994910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c44c3-c9c3-4d25-9e95-1c71bf7c46f1 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:26.033502 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a10926-7024-459e-a757-cef5141398de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:26.034675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a10926-7024-459e-a757-cef5141398de None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:26.034977 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a10926-7024-459e-a757-cef5141398de None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:26.035442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a10926-7024-459e-a757-cef5141398de None admin] PATH_INFO: `/v3/projects/service` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:26.042757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a10926-7024-459e-a757-cef5141398de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:26.044238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a10926-7024-459e-a757-cef5141398de None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a7a10926-7024-459e-a757-cef5141398de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:39:26.058647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:26.059804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:26.061019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:39:26.061019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:39:26.061019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:39:26.061019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:39:26.061019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:39:26.061019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:39:26.061019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:39:26.061019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:26.061019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 624/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:39:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:26.066008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68a9db-3f1b-4cc6-8bb5-1bcf95aa9677 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:26.178238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68a9db-3f1b-4cc6-8bb5-1bcf95aa9677 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:26.218623 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a44b6c-7c5f-4800-bb11-71631fe1a232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:26.219321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a44b6c-7c5f-4800-bb11-71631fe1a232 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:26.219602 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a44b6c-7c5f-4800-bb11-71631fe1a232 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:26.219873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a44b6c-7c5f-4800-bb11-71631fe1a232 None admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:26.220716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a44b6c-7c5f-4800-bb11-71631fe1a232 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:26.221029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a44b6c-7c5f-4800-bb11-71631fe1a232 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:26.222488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a44b6c-7c5f-4800-bb11-71631fe1a232 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:26.239849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 635/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:39:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:39:26.249415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a10926-7024-459e-a757-cef5141398de None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:26.281772 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:26.379454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a10926-7024-459e-a757-cef5141398de None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:26.421477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f739c-f7ae-47c5-aa4d-ad05fc3551c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:26.422144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f739c-f7ae-47c5-aa4d-ad05fc3551c2 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:26.422431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f739c-f7ae-47c5-aa4d-ad05fc3551c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:26.422818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f739c-f7ae-47c5-aa4d-ad05fc3551c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:26.423658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f739c-f7ae-47c5-aa4d-ad05fc3551c2 None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=eded06390a2744c9aa52f77e3bfc2db9) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:26.423992 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f739c-f7ae-47c5-aa4d-ad05fc3551c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:26.425656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f739c-f7ae-47c5-aa4d-ad05fc3551c2 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:26.432873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 625/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:39:26 2022] GET /identity/v3/role_assignments?user.id=eded06390a2744c9aa52f77e3bfc2db9&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:28.430825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97161b44-5dc1-4978-9104-0046d18fe276 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:28.431716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97161b44-5dc1-4978-9104-0046d18fe276 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:28.431716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97161b44-5dc1-4978-9104-0046d18fe276 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:28.433344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 636/1261] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:39:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:39:28.439515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788889f-218a-4488-8ec7-a588501af180 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:28.440247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788889f-218a-4488-8ec7-a588501af180 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:28.440435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8788889f-218a-4488-8ec7-a588501af180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:28.466406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8788889f-218a-4488-8ec7-a588501af180 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:39:28.467610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8788889f-218a-4488-8ec7-a588501af180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:39:28.537887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 626/1262] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:39:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:39:28.560227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97161b44-5dc1-4978-9104-0046d18fe276 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:28.571450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:28.670626 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97161b44-5dc1-4978-9104-0046d18fe276 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:28.713613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adffafb-5a2c-4ad8-8a5d-8c48dd849221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:28.714331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adffafb-5a2c-4ad8-8a5d-8c48dd849221 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:28.714609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adffafb-5a2c-4ad8-8a5d-8c48dd849221 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:28.714881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adffafb-5a2c-4ad8-8a5d-8c48dd849221 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:28.725570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adffafb-5a2c-4ad8-8a5d-8c48dd849221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-2adffafb-5a2c-4ad8-8a5d-8c48dd849221 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:39:28.733234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:28.734160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:28.734990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 637/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:39:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:28.740730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788889f-218a-4488-8ec7-a588501af180 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:28.759103 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:28.851372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8788889f-218a-4488-8ec7-a588501af180 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:28.889812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95936cb2-3e09-41de-9106-9ae0a9ed74aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:28.890762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95936cb2-3e09-41de-9106-9ae0a9ed74aa None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:28.891467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95936cb2-3e09-41de-9106-9ae0a9ed74aa None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:28.891812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95936cb2-3e09-41de-9106-9ae0a9ed74aa None admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:28.892998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95936cb2-3e09-41de-9106-9ae0a9ed74aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:28.893396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95936cb2-3e09-41de-9106-9ae0a9ed74aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:28.895540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95936cb2-3e09-41de-9106-9ae0a9ed74aa None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:28.909305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 627/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:39:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:28.918177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adffafb-5a2c-4ad8-8a5d-8c48dd849221 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:28.987819 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:29.023415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adffafb-5a2c-4ad8-8a5d-8c48dd849221 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:29.058783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc87c3d-ee33-4f91-ac67-0e126fd25a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:29.059587 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc87c3d-ee33-4f91-ac67-0e126fd25a9a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:29.059876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc87c3d-ee33-4f91-ac67-0e126fd25a9a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:29.060174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc87c3d-ee33-4f91-ac67-0e126fd25a9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:29.072389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc87c3d-ee33-4f91-ac67-0e126fd25a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:29.073917 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc87c3d-ee33-4f91-ac67-0e126fd25a9a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-4cc87c3d-ee33-4f91-ac67-0e126fd25a9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:29.092230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:29.093239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:29.094709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:29.094709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:29.094709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:29.094709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:29.094709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 638/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:29.102478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95936cb2-3e09-41de-9106-9ae0a9ed74aa None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:29.187819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:29.210754 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95936cb2-3e09-41de-9106-9ae0a9ed74aa None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:29.243660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f72f9f-0a77-421d-942d-af6b08660739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:29.244310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f72f9f-0a77-421d-942d-af6b08660739 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:29.244614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f72f9f-0a77-421d-942d-af6b08660739 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:29.244904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f72f9f-0a77-421d-942d-af6b08660739 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:29.245622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f72f9f-0a77-421d-942d-af6b08660739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:29.245924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f72f9f-0a77-421d-942d-af6b08660739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:29.247293 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f72f9f-0a77-421d-942d-af6b08660739 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:29.259178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 628/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:39:29.268445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc87c3d-ee33-4f91-ac67-0e126fd25a9a None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:29.377293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc87c3d-ee33-4f91-ac67-0e126fd25a9a None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:29.397309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:29.420334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede63d7-f8ef-438d-bbe3-da32470b4cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:29.421140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede63d7-f8ef-438d-bbe3-da32470b4cfe None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:29.421426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede63d7-f8ef-438d-bbe3-da32470b4cfe None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:29.421699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede63d7-f8ef-438d-bbe3-da32470b4cfe None admin] PATH_INFO: `/v3/users/placement` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:29.429756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede63d7-f8ef-438d-bbe3-da32470b4cfe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:29.431214 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede63d7-f8ef-438d-bbe3-da32470b4cfe None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-9ede63d7-f8ef-438d-bbe3-da32470b4cfe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:39:29.445924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:29.447032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:29.448248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:29.448248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:29.448248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:39:29.448248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:39:29.448248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:39:29.448248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:39:29.448248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:39:29.448248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:39:29.448248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 03:39:29.448248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:29.448248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 639/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:39:29.455895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f72f9f-0a77-421d-942d-af6b08660739 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:29.570282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f72f9f-0a77-421d-942d-af6b08660739 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:29.605528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a611da-f9b3-4725-b2d5-a01280a13c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:29.606227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a611da-f9b3-4725-b2d5-a01280a13c55 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:29.606503 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a611da-f9b3-4725-b2d5-a01280a13c55 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:29.606791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a611da-f9b3-4725-b2d5-a01280a13c55 None admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:29.607637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a611da-f9b3-4725-b2d5-a01280a13c55 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:29.607947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a611da-f9b3-4725-b2d5-a01280a13c55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:29.609457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a611da-f9b3-4725-b2d5-a01280a13c55 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:29.618439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:29.627910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 629/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Jul 12 03:39:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:29.635386 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede63d7-f8ef-438d-bbe3-da32470b4cfe None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:29.753998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede63d7-f8ef-438d-bbe3-da32470b4cfe None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:29.789470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df787df1-0735-4d80-9a6c-7b7d7faebc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:29.790249 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df787df1-0735-4d80-9a6c-7b7d7faebc90 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:29.790537 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df787df1-0735-4d80-9a6c-7b7d7faebc90 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:29.790812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df787df1-0735-4d80-9a6c-7b7d7faebc90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:29.801896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df787df1-0735-4d80-9a6c-7b7d7faebc90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:29.803378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df787df1-0735-4d80-9a6c-7b7d7faebc90 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-df787df1-0735-4d80-9a6c-7b7d7faebc90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:29.820773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:29.821758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:29.823062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:29.823062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:29.823062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:29.823062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:29.823062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 640/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:29.831047 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a611da-f9b3-4725-b2d5-a01280a13c55 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:29.939081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a611da-f9b3-4725-b2d5-a01280a13c55 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:29.977061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981af24-ea71-444d-8483-e486f47ccee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:29.977737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981af24-ea71-444d-8483-e486f47ccee6 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:29.978024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981af24-ea71-444d-8483-e486f47ccee6 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:29.978301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981af24-ea71-444d-8483-e486f47ccee6 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:29.979029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981af24-ea71-444d-8483-e486f47ccee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:29.979335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981af24-ea71-444d-8483-e486f47ccee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:29.980778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981af24-ea71-444d-8483-e486f47ccee6 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:29.992332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 630/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:29.998263 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df787df1-0735-4d80-9a6c-7b7d7faebc90 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:30.038041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:30.118877 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df787df1-0735-4d80-9a6c-7b7d7faebc90 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:30.161204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3010bc-a322-452e-b119-176090365a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:30.162049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3010bc-a322-452e-b119-176090365a64 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:30.162330 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3010bc-a322-452e-b119-176090365a64 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:30.162604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3010bc-a322-452e-b119-176090365a64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:30.169886 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3010bc-a322-452e-b119-176090365a64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:30.171349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3010bc-a322-452e-b119-176090365a64 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-5a3010bc-a322-452e-b119-176090365a64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:39:30.186353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:30.187369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:30.188400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:39:30.188400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:39:30.188400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:39:30.188400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:39:30.188400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:39:30.188400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:39:30.188400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:39:30.188400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:30.188400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 641/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:39:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:30.193558 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981af24-ea71-444d-8483-e486f47ccee6 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:30.256652 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:30.301289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981af24-ea71-444d-8483-e486f47ccee6 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:30.337910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741c327-3452-444e-88ac-50a10fdc0818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:30.338593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741c327-3452-444e-88ac-50a10fdc0818 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:30.338855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741c327-3452-444e-88ac-50a10fdc0818 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:30.339161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741c327-3452-444e-88ac-50a10fdc0818 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:30.339942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741c327-3452-444e-88ac-50a10fdc0818 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:30.340276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741c327-3452-444e-88ac-50a10fdc0818 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:30.341724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741c327-3452-444e-88ac-50a10fdc0818 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:30.353601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 631/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:39:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:39:30.362723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3010bc-a322-452e-b119-176090365a64 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:30.460753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:30.470798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3010bc-a322-452e-b119-176090365a64 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:30.506115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e980ed-3bea-4526-86f0-3fd27336c163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:30.507020 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e980ed-3bea-4526-86f0-3fd27336c163 None admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:30.507295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e980ed-3bea-4526-86f0-3fd27336c163 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:30.507566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e980ed-3bea-4526-86f0-3fd27336c163 None admin] PATH_INFO: `/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/eded06390a2744c9aa52f77e3bfc2db9/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:30.509642 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e980ed-3bea-4526-86f0-3fd27336c163 None admin] RBAC: Authorizing `identity:create_grant(project_id=48b17053daa84e4fa94840777cb1bee6, user_id=eded06390a2744c9aa52f77e3bfc2db9, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:30.511137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e980ed-3bea-4526-86f0-3fd27336c163 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:30.526762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 642/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Tue Jul 12 03:39:30 2022] PUT /identity/v3/projects/48b17053daa84e4fa94840777cb1bee6/users/eded06390a2744c9aa52f77e3bfc2db9/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:39:32.540077 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f449d0-059f-403b-bbec-b1e9db57fa91 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:32.540820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f449d0-059f-403b-bbec-b1e9db57fa91 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:32.540820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f449d0-059f-403b-bbec-b1e9db57fa91 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:32.542287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 632/1274] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:39:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:39:32.551423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463937f0-d023-433e-a9df-48cf5cba517d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:32.552152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463937f0-d023-433e-a9df-48cf5cba517d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:32.552342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463937f0-d023-433e-a9df-48cf5cba517d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:32.580429 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-463937f0-d023-433e-a9df-48cf5cba517d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:39:32.582094 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-463937f0-d023-433e-a9df-48cf5cba517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:39:32.662314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 643/1275] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:39:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:39:32.686941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f449d0-059f-403b-bbec-b1e9db57fa91 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:32.814909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f449d0-059f-403b-bbec-b1e9db57fa91 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:32.842661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:32.861018 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fc05d-8410-498a-8c8e-89e7f88b1313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:32.861876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fc05d-8410-498a-8c8e-89e7f88b1313 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:32.862156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fc05d-8410-498a-8c8e-89e7f88b1313 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:32.862426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fc05d-8410-498a-8c8e-89e7f88b1313 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:32.873156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fc05d-8410-498a-8c8e-89e7f88b1313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-7d2fc05d-8410-498a-8c8e-89e7f88b1313 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 03:39:32.881053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:32.882097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:32.882952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 633/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Tue Jul 12 03:39:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:32.889181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463937f0-d023-433e-a9df-48cf5cba517d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:32.980293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:33.000655 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463937f0-d023-433e-a9df-48cf5cba517d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:33.036690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855343e6-2d89-4a07-a41d-3bf866ace705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:33.037363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855343e6-2d89-4a07-a41d-3bf866ace705 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:33.037652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855343e6-2d89-4a07-a41d-3bf866ace705 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:33.037931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855343e6-2d89-4a07-a41d-3bf866ace705 None admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:33.038682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855343e6-2d89-4a07-a41d-3bf866ace705 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:33.038993 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855343e6-2d89-4a07-a41d-3bf866ace705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:33.040416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855343e6-2d89-4a07-a41d-3bf866ace705 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:33.048805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 644/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Tue Jul 12 03:39:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:33.058091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fc05d-8410-498a-8c8e-89e7f88b1313 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:33.185823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fc05d-8410-498a-8c8e-89e7f88b1313 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:33.220322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd138d-03d1-4d07-a23c-33bb6d32855f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:33.221111 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd138d-03d1-4d07-a23c-33bb6d32855f None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:33.221406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd138d-03d1-4d07-a23c-33bb6d32855f None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:33.221681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd138d-03d1-4d07-a23c-33bb6d32855f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:33.232452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd138d-03d1-4d07-a23c-33bb6d32855f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:33.233906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd138d-03d1-4d07-a23c-33bb6d32855f None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-a3bd138d-03d1-4d07-a23c-33bb6d32855f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:33.261835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:33.263032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:33.264320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:33.264320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:33.264320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:33.264320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:33.268287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 634/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:33.269923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855343e6-2d89-4a07-a41d-3bf866ace705 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:33.385619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855343e6-2d89-4a07-a41d-3bf866ace705 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:33.421283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd011b8-7417-4b9d-9935-08f623eaf848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:33.421955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd011b8-7417-4b9d-9935-08f623eaf848 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:33.422239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd011b8-7417-4b9d-9935-08f623eaf848 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:33.422520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd011b8-7417-4b9d-9935-08f623eaf848 None admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:33.423247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd011b8-7417-4b9d-9935-08f623eaf848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:33.423556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd011b8-7417-4b9d-9935-08f623eaf848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:33.424987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd011b8-7417-4b9d-9935-08f623eaf848 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:33.439203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 645/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:39:33.445124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd138d-03d1-4d07-a23c-33bb6d32855f None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:33.483340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:33.564658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd138d-03d1-4d07-a23c-33bb6d32855f None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:33.598403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e25e4c-6508-4d81-a972-4dd4b0131216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:33.599213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e25e4c-6508-4d81-a972-4dd4b0131216 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:33.599494 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e25e4c-6508-4d81-a972-4dd4b0131216 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:33.599766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e25e4c-6508-4d81-a972-4dd4b0131216 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:33.607251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e25e4c-6508-4d81-a972-4dd4b0131216 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:33.608722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e25e4c-6508-4d81-a972-4dd4b0131216 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-70e25e4c-6508-4d81-a972-4dd4b0131216 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 03:39:33.623084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:33.624211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:33.625351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:33.625351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:33.625351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 03:39:33.625351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 03:39:33.625351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 03:39:33.625351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 03:39:33.625351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 03:39:33.625351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 03:39:33.625351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 03:39:33.625351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:33.625351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 635/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:33.632606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd011b8-7417-4b9d-9935-08f623eaf848 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:33.638578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:33.735961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd011b8-7417-4b9d-9935-08f623eaf848 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:33.773998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab95c3-08e0-4626-b949-cf8a622c5e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:33.774660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab95c3-08e0-4626-b949-cf8a622c5e4d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:33.774939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab95c3-08e0-4626-b949-cf8a622c5e4d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:33.775212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab95c3-08e0-4626-b949-cf8a622c5e4d None admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:33.776118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab95c3-08e0-4626-b949-cf8a622c5e4d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:33.776427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab95c3-08e0-4626-b949-cf8a622c5e4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:33.777864 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab95c3-08e0-4626-b949-cf8a622c5e4d None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:33.797307 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 646/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Tue Jul 12 03:39:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:33.803316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e25e4c-6508-4d81-a972-4dd4b0131216 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:33.900732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:33.916655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e25e4c-6508-4d81-a972-4dd4b0131216 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:33.950878 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adbe7b-55cc-4f8b-85ee-43d6999db8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:33.951516 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adbe7b-55cc-4f8b-85ee-43d6999db8bd None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:33.951794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adbe7b-55cc-4f8b-85ee-43d6999db8bd None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:33.952092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adbe7b-55cc-4f8b-85ee-43d6999db8bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:34.022836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adbe7b-55cc-4f8b-85ee-43d6999db8bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:34.024291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adbe7b-55cc-4f8b-85ee-43d6999db8bd None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:34.119337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:34.131148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 636/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Tue Jul 12 03:39:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 03:39:34.139062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab95c3-08e0-4626-b949-cf8a622c5e4d None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:34.209107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:34.250259 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab95c3-08e0-4626-b949-cf8a622c5e4d None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:34.287362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac96fc1-9291-4fe8-b800-96ff6bf91a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:34.288114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac96fc1-9291-4fe8-b800-96ff6bf91a96 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:34.288394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac96fc1-9291-4fe8-b800-96ff6bf91a96 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:34.288678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac96fc1-9291-4fe8-b800-96ff6bf91a96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:34.298888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac96fc1-9291-4fe8-b800-96ff6bf91a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:34.300363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac96fc1-9291-4fe8-b800-96ff6bf91a96 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-2ac96fc1-9291-4fe8-b800-96ff6bf91a96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 03:39:34.317771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:34.318785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:34.319726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 03:39:34.319726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 03:39:34.319726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 03:39:34.319726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:34.319726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 647/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 03:39:34.324933 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adbe7b-55cc-4f8b-85ee-43d6999db8bd None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:34.429643 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adbe7b-55cc-4f8b-85ee-43d6999db8bd None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:34.469145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d5f64-9d65-44de-a9ba-3f7d6d142c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:34.470163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d5f64-9d65-44de-a9ba-3f7d6d142c35 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:34.470546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d5f64-9d65-44de-a9ba-3f7d6d142c35 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:34.470908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d5f64-9d65-44de-a9ba-3f7d6d142c35 None admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:34.472053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d5f64-9d65-44de-a9ba-3f7d6d142c35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:34.472484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d5f64-9d65-44de-a9ba-3f7d6d142c35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:34.474744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d5f64-9d65-44de-a9ba-3f7d6d142c35 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:34.493650 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 637/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Tue Jul 12 03:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:34.501468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac96fc1-9291-4fe8-b800-96ff6bf91a96 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:34.607002 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac96fc1-9291-4fe8-b800-96ff6bf91a96 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:34.621129 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:34.642224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34268f-f5ad-4bd7-9e9c-b2120368a516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:34.643010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34268f-f5ad-4bd7-9e9c-b2120368a516 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:34.643295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34268f-f5ad-4bd7-9e9c-b2120368a516 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:34.643573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34268f-f5ad-4bd7-9e9c-b2120368a516 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:34.650433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34268f-f5ad-4bd7-9e9c-b2120368a516 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:34.651882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34268f-f5ad-4bd7-9e9c-b2120368a516 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application [None req-9b34268f-f5ad-4bd7-9e9c-b2120368a516 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 03:39:34.665702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:34.666755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:34.667813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 03:39:34.667813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 03:39:34.667813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 03:39:34.667813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 03:39:34.667813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 03:39:34.667813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 03:39:34.667813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 03:39:34.667813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: ERROR keystone.server.flask.application Jul 12 03:39:34.667813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 648/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Tue Jul 12 03:39:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:34.672912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d5f64-9d65-44de-a9ba-3f7d6d142c35 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:34.729422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:34.784135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d5f64-9d65-44de-a9ba-3f7d6d142c35 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:34.821076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310809d-d041-4655-a241-3677bf2a4609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:34.821743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310809d-d041-4655-a241-3677bf2a4609 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:34.822024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310809d-d041-4655-a241-3677bf2a4609 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:34.822302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310809d-d041-4655-a241-3677bf2a4609 None admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:34.823162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310809d-d041-4655-a241-3677bf2a4609 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:34.823472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310809d-d041-4655-a241-3677bf2a4609 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:34.824955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310809d-d041-4655-a241-3677bf2a4609 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:34.837290 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 638/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Tue Jul 12 03:39:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:34.843416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34268f-f5ad-4bd7-9e9c-b2120368a516 None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:34.955438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34268f-f5ad-4bd7-9e9c-b2120368a516 None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:34.990731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccdc3c-4f07-48b5-a27f-7d19ead2ff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:34.991381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccdc3c-4f07-48b5-a27f-7d19ead2ff7d None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:34.991660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccdc3c-4f07-48b5-a27f-7d19ead2ff7d None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:34.991904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccdc3c-4f07-48b5-a27f-7d19ead2ff7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:34.992801 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccdc3c-4f07-48b5-a27f-7d19ead2ff7d None admin] RBAC: Adding query filter params (role.id=bef972566e4a4b3b97d844491e12ace2, scope.project.id=48b17053daa84e4fa94840777cb1bee6, user.id=eded06390a2744c9aa52f77e3bfc2db9) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:34.993111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccdc3c-4f07-48b5-a27f-7d19ead2ff7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:34.994560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccdc3c-4f07-48b5-a27f-7d19ead2ff7d None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:35.000589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 649/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Tue Jul 12 03:39:34 2022] GET /identity/v3/role_assignments?user.id=eded06390a2744c9aa52f77e3bfc2db9&scope.project.id=48b17053daa84e4fa94840777cb1bee6&role.id=bef972566e4a4b3b97d844491e12ace2 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:37.001797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349289dd-29ae-470f-9371-83604e62a114 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:37.002596 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349289dd-29ae-470f-9371-83604e62a114 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:37.002716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349289dd-29ae-470f-9371-83604e62a114 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:37.004942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 639/1288] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:39:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:39:37.011060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e655b5-18a8-420b-97b2-064203647ae5 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:37.011836 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e655b5-18a8-420b-97b2-064203647ae5 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:37.012043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e655b5-18a8-420b-97b2-064203647ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:37.030590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-62e655b5-18a8-420b-97b2-064203647ae5 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:39:37.031772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-62e655b5-18a8-420b-97b2-064203647ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:39:37.061594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:37.105104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 650/1289] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:39:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:39:37.140089 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349289dd-29ae-470f-9371-83604e62a114 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:37.252501 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349289dd-29ae-470f-9371-83604e62a114 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:37.263245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:37.288951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c32b9-64df-4fdd-9b8d-0f46c61ed385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:37.289738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c32b9-64df-4fdd-9b8d-0f46c61ed385 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:37.290017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c32b9-64df-4fdd-9b8d-0f46c61ed385 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:37.290289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c32b9-64df-4fdd-9b8d-0f46c61ed385 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:37.290940 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c32b9-64df-4fdd-9b8d-0f46c61ed385 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:37.292360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c32b9-64df-4fdd-9b8d-0f46c61ed385 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-fb0c32b9-64df-4fdd-9b8d-0f46c61ed385 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:37.304618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:37.305781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:37.306685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 640/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 12 03:39:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:37.311989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e655b5-18a8-420b-97b2-064203647ae5 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:37.420410 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e655b5-18a8-420b-97b2-064203647ae5 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:37.455560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8320f1-49f1-45df-9f83-c4bea64e3825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:37.456257 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8320f1-49f1-45df-9f83-c4bea64e3825 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:37.456541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8320f1-49f1-45df-9f83-c4bea64e3825 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:37.456810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8320f1-49f1-45df-9f83-c4bea64e3825 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:37.457515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8320f1-49f1-45df-9f83-c4bea64e3825 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:37.457826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8320f1-49f1-45df-9f83-c4bea64e3825 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:37.459222 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8320f1-49f1-45df-9f83-c4bea64e3825 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:37.466288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 651/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Jul 12 03:39:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:37.472262 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c32b9-64df-4fdd-9b8d-0f46c61ed385 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:37.576386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c32b9-64df-4fdd-9b8d-0f46c61ed385 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:37.612513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6279f8-b6e5-492d-9de2-5b307a6c3ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:37.613163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6279f8-b6e5-492d-9de2-5b307a6c3ded None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:37.613446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6279f8-b6e5-492d-9de2-5b307a6c3ded None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:37.613718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6279f8-b6e5-492d-9de2-5b307a6c3ded None admin] PATH_INFO: `/v3/services` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:37.614422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6279f8-b6e5-492d-9de2-5b307a6c3ded None admin] RBAC: Adding query filter params (type=placement) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:37.614728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6279f8-b6e5-492d-9de2-5b307a6c3ded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:37.616128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6279f8-b6e5-492d-9de2-5b307a6c3ded None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:37.620703 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 641/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Jul 12 03:39:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:39.599560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e76e75-8e5c-4f8a-8c3e-317f9be09ecf None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:39.600553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e76e75-8e5c-4f8a-8c3e-317f9be09ecf None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:39.600553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e76e75-8e5c-4f8a-8c3e-317f9be09ecf None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:39.603244 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 652/1293] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:39:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:39:39.609590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43b629-cc0f-4ea5-95d0-d6d68eb54dc9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:39.610455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43b629-cc0f-4ea5-95d0-d6d68eb54dc9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:39.610644 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43b629-cc0f-4ea5-95d0-d6d68eb54dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:39.632428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2c43b629-cc0f-4ea5-95d0-d6d68eb54dc9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:39:39.633671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2c43b629-cc0f-4ea5-95d0-d6d68eb54dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:39:39.670180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:39.734833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 642/1294] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:39:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:39:39.763955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e76e75-8e5c-4f8a-8c3e-317f9be09ecf None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:39.798152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:39.878670 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e76e75-8e5c-4f8a-8c3e-317f9be09ecf None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:39.915916 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91bc6c6-24c0-45ab-88a7-b80f7e1aa068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:39.916612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bc6c6-24c0-45ab-88a7-b80f7e1aa068 None admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:39.916888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bc6c6-24c0-45ab-88a7-b80f7e1aa068 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:39.917162 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91bc6c6-24c0-45ab-88a7-b80f7e1aa068 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:39.917973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91bc6c6-24c0-45ab-88a7-b80f7e1aa068 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:39.919334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91bc6c6-24c0-45ab-88a7-b80f7e1aa068 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:39.933923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 653/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Tue Jul 12 03:39:39 2022] POST /identity/v3/services => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:39:41.934639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce069b63-862d-4fcc-b609-9403a2280299 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:41.935562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce069b63-862d-4fcc-b609-9403a2280299 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:41.935562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce069b63-862d-4fcc-b609-9403a2280299 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:41.937731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 643/1296] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:39:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:39:41.946424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d141f3-1365-44b0-aca4-a07fa5ef850f None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:41.946860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d141f3-1365-44b0-aca4-a07fa5ef850f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:41.946996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d141f3-1365-44b0-aca4-a07fa5ef850f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:41.969998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-71d141f3-1365-44b0-aca4-a07fa5ef850f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:39:41.971236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-71d141f3-1365-44b0-aca4-a07fa5ef850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:39:42.053106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 654/1297] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:39:41 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:39:42.076623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce069b63-862d-4fcc-b609-9403a2280299 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:42.185921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce069b63-862d-4fcc-b609-9403a2280299 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:42.221384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fead84-dd5f-48e3-8bbd-57b69edbfb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:42.222182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fead84-dd5f-48e3-8bbd-57b69edbfb56 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:42.222467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fead84-dd5f-48e3-8bbd-57b69edbfb56 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:42.222742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fead84-dd5f-48e3-8bbd-57b69edbfb56 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:42.223399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fead84-dd5f-48e3-8bbd-57b69edbfb56 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:42.224815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fead84-dd5f-48e3-8bbd-57b69edbfb56 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-06fead84-dd5f-48e3-8bbd-57b69edbfb56 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:42.236948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:42.238214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:42.239306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 644/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 12 03:39:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:42.245423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d141f3-1365-44b0-aca4-a07fa5ef850f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:42.266084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:42.346755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d141f3-1365-44b0-aca4-a07fa5ef850f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:42.381138 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e26a8-47bd-4bfe-b4ca-f1830ecea8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:42.381811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e26a8-47bd-4bfe-b4ca-f1830ecea8cf None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:42.382088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e26a8-47bd-4bfe-b4ca-f1830ecea8cf None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:42.382358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e26a8-47bd-4bfe-b4ca-f1830ecea8cf None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:42.383066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e26a8-47bd-4bfe-b4ca-f1830ecea8cf None admin] RBAC: Adding query filter params (name=placement) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:42.383376 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e26a8-47bd-4bfe-b4ca-f1830ecea8cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:42.384781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e26a8-47bd-4bfe-b4ca-f1830ecea8cf None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:42.391653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 655/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Jul 12 03:39:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:42.398395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fead84-dd5f-48e3-8bbd-57b69edbfb56 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:42.438395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:42.509513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fead84-dd5f-48e3-8bbd-57b69edbfb56 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:42.554383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e49355-1dff-49a5-a774-af1cf742a7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:42.555071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e49355-1dff-49a5-a774-af1cf742a7ac None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:42.555350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e49355-1dff-49a5-a774-af1cf742a7ac None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:42.555631 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e49355-1dff-49a5-a774-af1cf742a7ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:42.556461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e49355-1dff-49a5-a774-af1cf742a7ac None admin] RBAC: Adding query filter params (interface=public, service_id=e71d696141b84d4abd7651da5113cc96, region_id=RegionOne) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:42.556776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e49355-1dff-49a5-a774-af1cf742a7ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:42.558213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e49355-1dff-49a5-a774-af1cf742a7ac None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:42.564149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 645/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Tue Jul 12 03:39:42 2022] GET /identity/v3/endpoints?service_id=e71d696141b84d4abd7651da5113cc96&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:42.573376 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e26a8-47bd-4bfe-b4ca-f1830ecea8cf None admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:42.667650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:42.692195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e26a8-47bd-4bfe-b4ca-f1830ecea8cf None admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:42.729889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5f828-306c-4199-87ea-f7994fc1467a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:42.730693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5f828-306c-4199-87ea-f7994fc1467a None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:42.730818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5f828-306c-4199-87ea-f7994fc1467a None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:42.731092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5f828-306c-4199-87ea-f7994fc1467a None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:42.731838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5f828-306c-4199-87ea-f7994fc1467a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:42.733218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5f828-306c-4199-87ea-f7994fc1467a None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:42.740550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 656/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Tue Jul 12 03:39:42 2022] GET /identity/v3/services => generated 2191 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:39:44.730260 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb2fe7-c4a2-4b2e-adea-ea92524d90c8 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:44.730801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb2fe7-c4a2-4b2e-adea-ea92524d90c8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:44.730801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb2fe7-c4a2-4b2e-adea-ea92524d90c8 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:44.732253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 646/1302] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:39:44.737579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d3292-660b-43c1-beaf-3dd745ccbc0c None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:44.737973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d3292-660b-43c1-beaf-3dd745ccbc0c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:44.738144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d3292-660b-43c1-beaf-3dd745ccbc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:44.756024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f86d3292-660b-43c1-beaf-3dd745ccbc0c None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:39:44.757379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f86d3292-660b-43c1-beaf-3dd745ccbc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:39:44.829974 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 657/1303] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:39:44 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 03:39:44.853296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb2fe7-c4a2-4b2e-adea-ea92524d90c8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:44.954949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:44.963003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb2fe7-c4a2-4b2e-adea-ea92524d90c8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:44.997811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b9f58-7277-49fe-9507-9ff9787ea504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:44.998541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b9f58-7277-49fe-9507-9ff9787ea504 None admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:44.998823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b9f58-7277-49fe-9507-9ff9787ea504 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:44.999098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b9f58-7277-49fe-9507-9ff9787ea504 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:44.999741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b9f58-7277-49fe-9507-9ff9787ea504 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:45.001143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b9f58-7277-49fe-9507-9ff9787ea504 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application [None req-8c7b9f58-7277-49fe-9507-9ff9787ea504 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 03:39:45.013057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 03:39:45.014169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application with Lock( Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self._enter() Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application return self.creator() Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application created_value = creator( Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: ERROR keystone.server.flask.application Jul 12 03:39:45.015579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 647/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Tue Jul 12 03:39:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 03:39:45.020812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d3292-660b-43c1-beaf-3dd745ccbc0c None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:45.101308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:45.125499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d3292-660b-43c1-beaf-3dd745ccbc0c None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:45.160507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c04c3-d1e7-49cf-9550-95749f85ffd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:45.161184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c04c3-d1e7-49cf-9550-95749f85ffd8 None admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:45.161478 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c04c3-d1e7-49cf-9550-95749f85ffd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:45.161754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c04c3-d1e7-49cf-9550-95749f85ffd8 None admin] PATH_INFO: `/v3/services` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:45.162458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c04c3-d1e7-49cf-9550-95749f85ffd8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:39:45.162766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c04c3-d1e7-49cf-9550-95749f85ffd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:45.164198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c04c3-d1e7-49cf-9550-95749f85ffd8 None admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:45.169620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 658/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Tue Jul 12 03:39:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:39:45.175660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b9f58-7277-49fe-9507-9ff9787ea504 None admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:39:45.280058 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b9f58-7277-49fe-9507-9ff9787ea504 None admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:39:45.312905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f089d-d354-4f8b-bdfb-e65b5c0f9948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:39:45.313548 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f089d-d354-4f8b-bdfb-e65b5c0f9948 None admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:39:45.313835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f089d-d354-4f8b-bdfb-e65b5c0f9948 None admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:39:45.314110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f089d-d354-4f8b-bdfb-e65b5c0f9948 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:39:45.314847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f089d-d354-4f8b-bdfb-e65b5c0f9948 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:39:45.316258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f089d-d354-4f8b-bdfb-e65b5c0f9948 None admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:39:45.335095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 648/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Tue Jul 12 03:39:45 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:39:55.166319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=30 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:39:55.330909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:40:48.325526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7221c92-797d-4d9a-b643-c333baa8a09d None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:48.326012 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7221c92-797d-4d9a-b643-c333baa8a09d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:48.326091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7221c92-797d-4d9a-b643-c333baa8a09d None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:48.327465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 659/1307] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:40:48.332768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2d09b-0ce7-434d-8d00-674041d53517 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:48.333162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2d09b-0ce7-434d-8d00-674041d53517 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:48.333299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2d09b-0ce7-434d-8d00-674041d53517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:48.427330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-77e2d09b-0ce7-434d-8d00-674041d53517 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:40:48.429247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-77e2d09b-0ce7-434d-8d00-674041d53517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:48.576633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 649/1308] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:40:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:40:48.639063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfbde1-6f1e-4dce-88f9-ce927143e9a7 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:48.639390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfbde1-6f1e-4dce-88f9-ce927143e9a7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:48.639770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfbde1-6f1e-4dce-88f9-ce927143e9a7 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:48.643178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 660/1309] 10.222.0.20 () {54 vars in 964 bytes} [Tue Jul 12 03:40:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:40:48.651081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e02f14-576d-498e-bb3a-264759b4a512 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:48.651508 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e02f14-576d-498e-bb3a-264759b4a512 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:48.651796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e02f14-576d-498e-bb3a-264759b4a512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:48.744485 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-90e02f14-576d-498e-bb3a-264759b4a512 None None] MFA Rules not processed for user `139be21e7f644b5796d30158851ebd00`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:40:48.745988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-90e02f14-576d-498e-bb3a-264759b4a512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:48.770435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:40:48.840888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfbde1-6f1e-4dce-88f9-ce927143e9a7 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:40:48.842969 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b5cfbde1-6f1e-4dce-88f9-ce927143e9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:48.846142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 650/1310] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Jul 12 03:40:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:40:48.978574 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfbde1-6f1e-4dce-88f9-ce927143e9a7 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:40:49.064261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9c21d-8659-478e-b50f-3855a86d5320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:40:49.065025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9c21d-8659-478e-b50f-3855a86d5320 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:49.065382 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9c21d-8659-478e-b50f-3855a86d5320 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:49.065729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9c21d-8659-478e-b50f-3855a86d5320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:49.067584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-dac9c21d-8659-478e-b50f-3855a86d5320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:49.124383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:40:49.184982 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9c21d-8659-478e-b50f-3855a86d5320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:40:49.187180 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac9c21d-8659-478e-b50f-3855a86d5320 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:40:49.207668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 661/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:40:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:40:51.582720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b3795-bcef-41d8-8197-927285258103 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:51.583312 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b3795-bcef-41d8-8197-927285258103 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:51.583312 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b3795-bcef-41d8-8197-927285258103 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:51.584765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 651/1312] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:40:51.593751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8512b0-b38c-46b1-9d52-ec2f18cbf30f None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:51.594589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8512b0-b38c-46b1-9d52-ec2f18cbf30f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:51.595104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8512b0-b38c-46b1-9d52-ec2f18cbf30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:51.679355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-dd8512b0-b38c-46b1-9d52-ec2f18cbf30f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:40:51.680615 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-dd8512b0-b38c-46b1-9d52-ec2f18cbf30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:51.788313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 662/1313] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:40:51.827402 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b3795-bcef-41d8-8197-927285258103 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:40:51.829092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dc3b3795-bcef-41d8-8197-927285258103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:51.944503 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b3795-bcef-41d8-8197-927285258103 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:40:52.021652 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:40:52.024587 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f082503-7f80-4455-8af1-fc7bb9b40596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:40:52.025262 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f082503-7f80-4455-8af1-fc7bb9b40596 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:52.025539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f082503-7f80-4455-8af1-fc7bb9b40596 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:52.025820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f082503-7f80-4455-8af1-fc7bb9b40596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:52.027904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2f082503-7f80-4455-8af1-fc7bb9b40596 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:52.145948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f082503-7f80-4455-8af1-fc7bb9b40596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:40:52.147424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f082503-7f80-4455-8af1-fc7bb9b40596 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:40:52.167583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 652/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:40:54.516483 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56719a-67be-401a-be0e-5cc344ef5ffb None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:54.517085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56719a-67be-401a-be0e-5cc344ef5ffb None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:54.517085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56719a-67be-401a-be0e-5cc344ef5ffb None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:54.518416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 663/1315] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:40:54.527179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e982e47-6655-4e03-b8ac-3e92f1b69b9d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:54.527946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e982e47-6655-4e03-b8ac-3e92f1b69b9d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:54.528169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e982e47-6655-4e03-b8ac-3e92f1b69b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:54.614443 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3e982e47-6655-4e03-b8ac-3e92f1b69b9d None None] MFA Rules not processed for user `ea069f8de53148a798edcfd4984feff3`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:40:54.615723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3e982e47-6655-4e03-b8ac-3e92f1b69b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:54.669726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:40:54.715694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 653/1316] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:40:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:40:54.764038 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56719a-67be-401a-be0e-5cc344ef5ffb None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:40:54.765631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3b56719a-67be-401a-be0e-5cc344ef5ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:54.776709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:40:54.879654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56719a-67be-401a-be0e-5cc344ef5ffb None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:40:54.958441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78359bf-4dc5-4748-99e4-cacc2b59bfd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:40:54.959116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78359bf-4dc5-4748-99e4-cacc2b59bfd5 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:54.959399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78359bf-4dc5-4748-99e4-cacc2b59bfd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:54.959677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78359bf-4dc5-4748-99e4-cacc2b59bfd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:54.961359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b78359bf-4dc5-4748-99e4-cacc2b59bfd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:55.026154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:40:55.075290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78359bf-4dc5-4748-99e4-cacc2b59bfd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:40:55.076792 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78359bf-4dc5-4748-99e4-cacc2b59bfd5 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:40:55.095434 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 664/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:40:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:40:57.680541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e2686-d4c1-40cf-803a-1b31d401f456 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:57.681114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e2686-d4c1-40cf-803a-1b31d401f456 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:57.681114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e2686-d4c1-40cf-803a-1b31d401f456 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:57.683130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 654/1318] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:40:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:40:57.688897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea498b-6f68-40ed-b1b5-b5ed4414700e None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:57.689305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea498b-6f68-40ed-b1b5-b5ed4414700e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:57.689450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea498b-6f68-40ed-b1b5-b5ed4414700e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:57.758752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-41ea498b-6f68-40ed-b1b5-b5ed4414700e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:40:57.760420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-41ea498b-6f68-40ed-b1b5-b5ed4414700e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:57.860174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 665/1319] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:40:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:40:57.905819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e2686-d4c1-40cf-803a-1b31d401f456 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:40:57.908256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d28e2686-d4c1-40cf-803a-1b31d401f456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:58.020400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e2686-d4c1-40cf-803a-1b31d401f456 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:40:58.095052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244b9d9-99b6-4cf4-a487-d7ba3b834653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:40:58.095695 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244b9d9-99b6-4cf4-a487-d7ba3b834653 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:40:58.095941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244b9d9-99b6-4cf4-a487-d7ba3b834653 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:40:58.096291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244b9d9-99b6-4cf4-a487-d7ba3b834653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:40:58.097914 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7244b9d9-99b6-4cf4-a487-d7ba3b834653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:40:58.105514 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:40:58.210518 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244b9d9-99b6-4cf4-a487-d7ba3b834653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:40:58.212073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244b9d9-99b6-4cf4-a487-d7ba3b834653 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:40:58.230564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 655/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:40:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:00.921944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ffb3-0e53-4456-a172-d8311f9a8247 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:00.922550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ffb3-0e53-4456-a172-d8311f9a8247 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:00.922550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ffb3-0e53-4456-a172-d8311f9a8247 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:00.923863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 666/1321] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:00.929279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a33c1-18e7-4c2b-942f-e07ac69ffe52 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:00.929712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a33c1-18e7-4c2b-942f-e07ac69ffe52 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:00.929826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a33c1-18e7-4c2b-942f-e07ac69ffe52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:00.999037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9c4a33c1-18e7-4c2b-942f-e07ac69ffe52 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:01.000273 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9c4a33c1-18e7-4c2b-942f-e07ac69ffe52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:01.058659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:01.106190 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 656/1322] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:01.143137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0ffb3-0e53-4456-a172-d8311f9a8247 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:01.144775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-46e0ffb3-0e53-4456-a172-d8311f9a8247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:01.166007 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:01.258207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0ffb3-0e53-4456-a172-d8311f9a8247 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:01.331059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4381b-5832-40f0-8116-bf9275fb8b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:01.331712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4381b-5832-40f0-8116-bf9275fb8b7d service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:01.331995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4381b-5832-40f0-8116-bf9275fb8b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:01.332296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4381b-5832-40f0-8116-bf9275fb8b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:01.333999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-61c4381b-5832-40f0-8116-bf9275fb8b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:01.408560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:01.447100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4381b-5832-40f0-8116-bf9275fb8b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:01.448601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4381b-5832-40f0-8116-bf9275fb8b7d service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:01.467196 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 667/1323] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:03.541733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bb038-2ae7-4543-ad63-8f2f1a2f12b3 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:03.542292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bb038-2ae7-4543-ad63-8f2f1a2f12b3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:03.542292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3bb038-2ae7-4543-ad63-8f2f1a2f12b3 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:03.543548 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 657/1324] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:03.548921 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9bee3-16ea-4c4f-b3a4-2f894775af5d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:03.549378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9bee3-16ea-4c4f-b3a4-2f894775af5d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:03.549489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9bee3-16ea-4c4f-b3a4-2f894775af5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:03.618009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-55a9bee3-16ea-4c4f-b3a4-2f894775af5d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:03.620233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-55a9bee3-16ea-4c4f-b3a4-2f894775af5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:03.721609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 668/1325] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:03.772399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5ec8b-dfca-4fbd-bd7c-31a1d09f793c None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:03.772672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5ec8b-dfca-4fbd-bd7c-31a1d09f793c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:03.772977 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5ec8b-dfca-4fbd-bd7c-31a1d09f793c None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:03.774345 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 658/1326] 10.222.0.20 () {54 vars in 964 bytes} [Tue Jul 12 03:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:03.780350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b511f-464a-4d08-9943-073419abb817 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:03.780645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b511f-464a-4d08-9943-073419abb817 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:03.780927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b511f-464a-4d08-9943-073419abb817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:03.814112 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:03.862537 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-e27b511f-464a-4d08-9943-073419abb817 None None] MFA Rules not processed for user `139be21e7f644b5796d30158851ebd00`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:03.864132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e27b511f-464a-4d08-9943-073419abb817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:03.948671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 669/1327] 10.222.0.20 () {58 vars in 1061 bytes} [Tue Jul 12 03:41:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:03.957740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5ec8b-dfca-4fbd-bd7c-31a1d09f793c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:03.959303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-00e5ec8b-dfca-4fbd-bd7c-31a1d09f793c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:04.067346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5ec8b-dfca-4fbd-bd7c-31a1d09f793c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:04.148064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5a643-ba71-4c06-a7d8-b90ed143283e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:04.148741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5a643-ba71-4c06-a7d8-b90ed143283e service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:04.149031 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5a643-ba71-4c06-a7d8-b90ed143283e service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:04.149307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5a643-ba71-4c06-a7d8-b90ed143283e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:04.150927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-38b5a643-ba71-4c06-a7d8-b90ed143283e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:04.168617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:04.265085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5a643-ba71-4c06-a7d8-b90ed143283e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:04.266543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5a643-ba71-4c06-a7d8-b90ed143283e service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:04.285425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 659/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:06.576694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907cd39-bd0d-4b3f-9b16-4927799e03ee None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:06.577253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907cd39-bd0d-4b3f-9b16-4927799e03ee None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:06.577253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907cd39-bd0d-4b3f-9b16-4927799e03ee None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:06.578706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 670/1329] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:06.584481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322071b5-7c66-4ea6-bbc9-639582d079a4 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:06.584932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322071b5-7c66-4ea6-bbc9-639582d079a4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:06.585035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322071b5-7c66-4ea6-bbc9-639582d079a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:06.652218 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-322071b5-7c66-4ea6-bbc9-639582d079a4 None None] MFA Rules not processed for user `ea069f8de53148a798edcfd4984feff3`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:06.653600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-322071b5-7c66-4ea6-bbc9-639582d079a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:06.736645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:06.770595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 660/1330] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:06.809008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907cd39-bd0d-4b3f-9b16-4927799e03ee None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:06.810549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a907cd39-bd0d-4b3f-9b16-4927799e03ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:06.926202 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907cd39-bd0d-4b3f-9b16-4927799e03ee None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:06.947065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:07.004035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc073b-f610-470e-9873-00467ae5678a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:07.004756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc073b-f610-470e-9873-00467ae5678a service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:07.005039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc073b-f610-470e-9873-00467ae5678a service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:07.005319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc073b-f610-470e-9873-00467ae5678a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:07.006990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f8dc073b-f610-470e-9873-00467ae5678a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:07.118669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc073b-f610-470e-9873-00467ae5678a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:07.120162 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc073b-f610-470e-9873-00467ae5678a service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:07.139760 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 671/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:09.411453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0491fe4-0357-4989-9ff8-f1cb9a071a2c None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:09.412088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0491fe4-0357-4989-9ff8-f1cb9a071a2c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:09.412088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0491fe4-0357-4989-9ff8-f1cb9a071a2c None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:09.413419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 661/1332] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:09.418898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835caea6-cb99-4895-baf9-edca71337d71 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:09.419355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835caea6-cb99-4895-baf9-edca71337d71 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:09.419541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835caea6-cb99-4895-baf9-edca71337d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:09.513616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:09.516386 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-835caea6-cb99-4895-baf9-edca71337d71 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:09.518284 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-835caea6-cb99-4895-baf9-edca71337d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:09.630332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 672/1333] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:09.671864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0491fe4-0357-4989-9ff8-f1cb9a071a2c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:09.673647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c0491fe4-0357-4989-9ff8-f1cb9a071a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:09.800808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0491fe4-0357-4989-9ff8-f1cb9a071a2c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:09.883852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7eb1b2-a59b-4bd9-97cd-ccfc54a71845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:09.884556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7eb1b2-a59b-4bd9-97cd-ccfc54a71845 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:09.884837 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7eb1b2-a59b-4bd9-97cd-ccfc54a71845 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:09.885108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7eb1b2-a59b-4bd9-97cd-ccfc54a71845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:09.886774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5f7eb1b2-a59b-4bd9-97cd-ccfc54a71845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:09.914321 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:10.002077 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7eb1b2-a59b-4bd9-97cd-ccfc54a71845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:10.003537 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7eb1b2-a59b-4bd9-97cd-ccfc54a71845 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:10.022323 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 662/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:12.055458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46274b-1cb1-4a96-a3a9-1188e21c0ef5 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:12.056416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46274b-1cb1-4a96-a3a9-1188e21c0ef5 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:12.057000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46274b-1cb1-4a96-a3a9-1188e21c0ef5 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:12.061388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 673/1335] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:12.069601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd10f0-b5cc-4518-b61e-c920ee4dd73e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:12.070007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd10f0-b5cc-4518-b61e-c920ee4dd73e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:12.070154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd10f0-b5cc-4518-b61e-c920ee4dd73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:12.144286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1bfd10f0-b5cc-4518-b61e-c920ee4dd73e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:12.145484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1bfd10f0-b5cc-4518-b61e-c920ee4dd73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:12.223629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:12.257752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 663/1336] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:12.297604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46274b-1cb1-4a96-a3a9-1188e21c0ef5 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:12.299187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-9e46274b-1cb1-4a96-a3a9-1188e21c0ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:12.418389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46274b-1cb1-4a96-a3a9-1188e21c0ef5 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:12.447308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:12.494598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fc0c7-3c3f-42b0-847b-c89c81aaa5a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:12.495169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fc0c7-3c3f-42b0-847b-c89c81aaa5a0 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:12.495452 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fc0c7-3c3f-42b0-847b-c89c81aaa5a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:12.495722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fc0c7-3c3f-42b0-847b-c89c81aaa5a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:12.497389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ac9fc0c7-3c3f-42b0-847b-c89c81aaa5a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:12.619594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fc0c7-3c3f-42b0-847b-c89c81aaa5a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:12.621132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fc0c7-3c3f-42b0-847b-c89c81aaa5a0 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:12.641235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 674/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:15.036616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e4620-7c22-4e7b-930c-e74772453861 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:15.037200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e4620-7c22-4e7b-930c-e74772453861 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:15.037200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e4620-7c22-4e7b-930c-e74772453861 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:15.038509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 664/1338] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:15.044341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e19b5-4691-4f80-b832-302270827009 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:15.044787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e19b5-4691-4f80-b832-302270827009 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:15.044894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e19b5-4691-4f80-b832-302270827009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:15.111324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d66e19b5-4691-4f80-b832-302270827009 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:15.112606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d66e19b5-4691-4f80-b832-302270827009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:15.121009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:15.216997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 675/1339] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:41:15.257521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e4620-7c22-4e7b-930c-e74772453861 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:15.260306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-846e4620-7c22-4e7b-930c-e74772453861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:15.372608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e4620-7c22-4e7b-930c-e74772453861 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:15.451746 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c477af8-44bc-42e4-955e-50ad9fc413af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:15.452434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c477af8-44bc-42e4-955e-50ad9fc413af service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:15.452715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c477af8-44bc-42e4-955e-50ad9fc413af service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:15.452996 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c477af8-44bc-42e4-955e-50ad9fc413af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:15.454640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2c477af8-44bc-42e4-955e-50ad9fc413af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:15.492283 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:15.570528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c477af8-44bc-42e4-955e-50ad9fc413af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:15.571981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c477af8-44bc-42e4-955e-50ad9fc413af service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:15.590979 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 665/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:19.549209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e5d48-cc92-4bc9-8f16-111e061ef366 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:19.549809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e5d48-cc92-4bc9-8f16-111e061ef366 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:19.549809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e5d48-cc92-4bc9-8f16-111e061ef366 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:19.551155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 676/1341] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:19.556712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dd6b8-0a6b-49c2-9a63-36142bc83e96 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:19.557127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dd6b8-0a6b-49c2-9a63-36142bc83e96 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:19.557386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dd6b8-0a6b-49c2-9a63-36142bc83e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:19.625744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3f8dd6b8-0a6b-49c2-9a63-36142bc83e96 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:19.627140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3f8dd6b8-0a6b-49c2-9a63-36142bc83e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:19.717230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:19.729165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 666/1342] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:19.767290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e5d48-cc92-4bc9-8f16-111e061ef366 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:19.768888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6a8e5d48-cc92-4bc9-8f16-111e061ef366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:19.876391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e5d48-cc92-4bc9-8f16-111e061ef366 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:19.914720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:19.953307 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503bd282-d7ef-470c-a8a9-188a0ff7a6ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:19.953968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bd282-d7ef-470c-a8a9-188a0ff7a6ff service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:19.954248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bd282-d7ef-470c-a8a9-188a0ff7a6ff service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:19.954518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503bd282-d7ef-470c-a8a9-188a0ff7a6ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:19.956193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-503bd282-d7ef-470c-a8a9-188a0ff7a6ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:20.073607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bd282-d7ef-470c-a8a9-188a0ff7a6ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:20.075058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503bd282-d7ef-470c-a8a9-188a0ff7a6ff service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:20.093955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 677/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:23.078840 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118225f-6674-4b43-b9c9-26c9e26b6383 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:23.079407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118225f-6674-4b43-b9c9-26c9e26b6383 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:23.079407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118225f-6674-4b43-b9c9-26c9e26b6383 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:23.080774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 667/1344] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:23.086580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cad19-a8d1-463d-8184-e2f707848afa None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:23.087019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cad19-a8d1-463d-8184-e2f707848afa None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:23.087118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cad19-a8d1-463d-8184-e2f707848afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:23.153908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-de9cad19-a8d1-463d-8184-e2f707848afa None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:23.155107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-de9cad19-a8d1-463d-8184-e2f707848afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:23.174354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:23.256199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 678/1345] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:23.293568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118225f-6674-4b43-b9c9-26c9e26b6383 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:23.295113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c118225f-6674-4b43-b9c9-26c9e26b6383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:23.402357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118225f-6674-4b43-b9c9-26c9e26b6383 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:23.481480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fd9cf-1c9c-4219-97fe-fb4ed3ac1fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:23.482210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fd9cf-1c9c-4219-97fe-fb4ed3ac1fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:23.482463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fd9cf-1c9c-4219-97fe-fb4ed3ac1fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:23.482764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fd9cf-1c9c-4219-97fe-fb4ed3ac1fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:23.484486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a73fd9cf-1c9c-4219-97fe-fb4ed3ac1fe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:23.528359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:23.599758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73fd9cf-1c9c-4219-97fe-fb4ed3ac1fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:23.601263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73fd9cf-1c9c-4219-97fe-fb4ed3ac1fe0 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:23.620591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 668/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:27.906977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b346a4-6a70-4743-9461-b2cf3493a949 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:27.907564 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b346a4-6a70-4743-9461-b2cf3493a949 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:27.907564 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b346a4-6a70-4743-9461-b2cf3493a949 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:27.909108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 679/1347] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:27.914445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71df10-7905-42a0-832f-46cf77a92ac9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:27.914968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71df10-7905-42a0-832f-46cf77a92ac9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:27.914968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71df10-7905-42a0-832f-46cf77a92ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:27.988677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7a71df10-7905-42a0-832f-46cf77a92ac9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:27.989910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7a71df10-7905-42a0-832f-46cf77a92ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:28.091837 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:28.094770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 669/1348] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:28.132512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b346a4-6a70-4743-9461-b2cf3493a949 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:28.134440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a0b346a4-6a70-4743-9461-b2cf3493a949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:28.243829 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b346a4-6a70-4743-9461-b2cf3493a949 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:28.290411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:28.316302 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bbead-b492-443e-a760-e8dbf5f0a2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:28.316961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bbead-b492-443e-a760-e8dbf5f0a2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:28.317242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bbead-b492-443e-a760-e8dbf5f0a2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:28.317504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bbead-b492-443e-a760-e8dbf5f0a2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:28.319135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-511bbead-b492-443e-a760-e8dbf5f0a2c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:28.430417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511bbead-b492-443e-a760-e8dbf5f0a2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:28.431907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511bbead-b492-443e-a760-e8dbf5f0a2c2 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:28.452820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 680/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:33.335951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11b32e-5e76-4f67-a648-b8fb05a08ed0 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:33.336408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11b32e-5e76-4f67-a648-b8fb05a08ed0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:33.336521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11b32e-5e76-4f67-a648-b8fb05a08ed0 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:33.338317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 670/1350] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:33.343086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328a29f-239f-4d8d-9315-62c676ab561a None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:33.343496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328a29f-239f-4d8d-9315-62c676ab561a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:33.343732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328a29f-239f-4d8d-9315-62c676ab561a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:33.439015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f328a29f-239f-4d8d-9315-62c676ab561a None None] MFA Rules not processed for user `eded06390a2744c9aa52f77e3bfc2db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:33.439015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f328a29f-239f-4d8d-9315-62c676ab561a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:33.463966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:33.575485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 681/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:41:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:37.739777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82329108-7ba4-4c7a-b2fa-218e356d4b2a None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:37.740470 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82329108-7ba4-4c7a-b2fa-218e356d4b2a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:37.740647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82329108-7ba4-4c7a-b2fa-218e356d4b2a None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:37.742310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 671/1352] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:37.747684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcf6f1-07c2-4bfb-b121-deacec106dd6 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:37.748583 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcf6f1-07c2-4bfb-b121-deacec106dd6 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:37.749031 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcf6f1-07c2-4bfb-b121-deacec106dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:37.843128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-16fcf6f1-07c2-4bfb-b121-deacec106dd6 None None] MFA Rules not processed for user `eded06390a2744c9aa52f77e3bfc2db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:37.845228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-16fcf6f1-07c2-4bfb-b121-deacec106dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:37.932810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:37.984510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 682/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:41:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:39.227521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18fd61-a78d-4a87-9333-b1ba9ce67f0b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:39.228064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18fd61-a78d-4a87-9333-b1ba9ce67f0b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:39.228064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18fd61-a78d-4a87-9333-b1ba9ce67f0b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:39.229504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 672/1354] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:39.234903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fbe31-af76-4174-a5bd-7c606631c854 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:39.235344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fbe31-af76-4174-a5bd-7c606631c854 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:39.235569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fbe31-af76-4174-a5bd-7c606631c854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:39.320219 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-3c4fbe31-af76-4174-a5bd-7c606631c854 None None] MFA Rules not processed for user `eded06390a2744c9aa52f77e3bfc2db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:39.321426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3c4fbe31-af76-4174-a5bd-7c606631c854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:39.432300 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 683/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:41:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:41:42.857881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ff3dc-3a19-4390-8840-d147c1304bdf None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:42.859719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ff3dc-3a19-4390-8840-d147c1304bdf None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:42.860353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ff3dc-3a19-4390-8840-d147c1304bdf None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:42.864636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 673/1356] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:42.868244 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6e3fa-feaa-45e8-b108-228b0dfcee86 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:42.868706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6e3fa-feaa-45e8-b108-228b0dfcee86 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:42.868818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6e3fa-feaa-45e8-b108-228b0dfcee86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:42.911615 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:42.950948 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-efc6e3fa-feaa-45e8-b108-228b0dfcee86 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:42.954244 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-efc6e3fa-feaa-45e8-b108-228b0dfcee86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:43.064084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 684/1357] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:43.114819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296da32-c570-49fa-b9db-ff89c51b106d None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:43.115162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296da32-c570-49fa-b9db-ff89c51b106d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:43.115418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296da32-c570-49fa-b9db-ff89c51b106d None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:43.116711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 674/1358] 10.222.0.20 () {54 vars in 961 bytes} [Tue Jul 12 03:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:43.125649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5eeee-dff7-401b-a528-e3c608198e08 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:43.126445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5eeee-dff7-401b-a528-e3c608198e08 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:43.127203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5eeee-dff7-401b-a528-e3c608198e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:43.216334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-29f5eeee-dff7-401b-a528-e3c608198e08 None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:43.217662 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-29f5eeee-dff7-401b-a528-e3c608198e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:43.270296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:43.344832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 685/1359] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Jul 12 03:41:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:41:43.352584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296da32-c570-49fa-b9db-ff89c51b106d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:43.354121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8296da32-c570-49fa-b9db-ff89c51b106d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:43.540069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296da32-c570-49fa-b9db-ff89c51b106d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:43.628779 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:43.676552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb7ce7-a703-4da4-885e-f427306ce197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:43.677764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb7ce7-a703-4da4-885e-f427306ce197 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:43.678053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb7ce7-a703-4da4-885e-f427306ce197 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:43.678330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb7ce7-a703-4da4-885e-f427306ce197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:43.681166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-45cb7ce7-a703-4da4-885e-f427306ce197 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:43.783003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb7ce7-a703-4da4-885e-f427306ce197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:43.785129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb7ce7-a703-4da4-885e-f427306ce197 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:43.806777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 675/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:46.062237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931e21c-9db4-468f-8aa1-a1f4e0e41e77 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:46.062902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931e21c-9db4-468f-8aa1-a1f4e0e41e77 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:46.062902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931e21c-9db4-468f-8aa1-a1f4e0e41e77 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:46.064471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 686/1361] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:46.069769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75888a-9855-4a50-94a6-fd0e0a819c45 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:46.070234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75888a-9855-4a50-94a6-fd0e0a819c45 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:46.070375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75888a-9855-4a50-94a6-fd0e0a819c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:46.140990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ae75888a-9855-4a50-94a6-fd0e0a819c45 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:46.142228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ae75888a-9855-4a50-94a6-fd0e0a819c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:46.181742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:46.253188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 676/1362] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:46.297007 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931e21c-9db4-468f-8aa1-a1f4e0e41e77 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:46.301191 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d931e21c-9db4-468f-8aa1-a1f4e0e41e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:46.514968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931e21c-9db4-468f-8aa1-a1f4e0e41e77 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:46.525674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:46.639811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00a855-1e05-46a4-b392-ea86f391f419 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:46.640489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00a855-1e05-46a4-b392-ea86f391f419 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:46.640803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00a855-1e05-46a4-b392-ea86f391f419 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:46.641057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00a855-1e05-46a4-b392-ea86f391f419 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:46.642718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6b00a855-1e05-46a4-b392-ea86f391f419 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:46.738095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00a855-1e05-46a4-b392-ea86f391f419 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:46.739757 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00a855-1e05-46a4-b392-ea86f391f419 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:46.759722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 687/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:49.096041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251910a-d8d4-4478-84e1-71087de7050b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:49.096769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251910a-d8d4-4478-84e1-71087de7050b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:49.096769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251910a-d8d4-4478-84e1-71087de7050b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:49.102139 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 677/1364] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:49.103759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1c80c-f690-4443-99c4-1730c4864494 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:49.104227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1c80c-f690-4443-99c4-1730c4864494 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:49.104386 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1c80c-f690-4443-99c4-1730c4864494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:49.124957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:49.182569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-0fa1c80c-f690-4443-99c4-1730c4864494 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:49.183906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0fa1c80c-f690-4443-99c4-1730c4864494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:49.292124 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 688/1365] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:49.338128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c41e3-51d5-433a-8ba2-78cf5b5a9698 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:49.338421 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c41e3-51d5-433a-8ba2-78cf5b5a9698 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:49.338701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c41e3-51d5-433a-8ba2-78cf5b5a9698 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:49.340087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 678/1366] 10.222.0.20 () {54 vars in 961 bytes} [Tue Jul 12 03:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:49.345521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebac288-c5bc-4490-a247-147527c6db75 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:49.345816 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebac288-c5bc-4490-a247-147527c6db75 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:49.346100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebac288-c5bc-4490-a247-147527c6db75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:49.419628 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-5ebac288-c5bc-4490-a247-147527c6db75 None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:49.420859 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5ebac288-c5bc-4490-a247-147527c6db75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:49.457798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:49.548164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 689/1367] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Jul 12 03:41:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:49.556297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c41e3-51d5-433a-8ba2-78cf5b5a9698 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:49.558076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0d5c41e3-51d5-433a-8ba2-78cf5b5a9698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:49.724270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c41e3-51d5-433a-8ba2-78cf5b5a9698 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:49.746543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:49.843078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40d3ef-ec5b-4d73-82fb-e505798509dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:49.843750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40d3ef-ec5b-4d73-82fb-e505798509dd service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:49.844027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40d3ef-ec5b-4d73-82fb-e505798509dd service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:49.844326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40d3ef-ec5b-4d73-82fb-e505798509dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:49.845961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ed40d3ef-ec5b-4d73-82fb-e505798509dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:49.944370 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40d3ef-ec5b-4d73-82fb-e505798509dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:49.946004 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40d3ef-ec5b-4d73-82fb-e505798509dd service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:49.970965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 679/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:50.888760 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb340bf1-52da-4f64-9943-18cdaa0b4c69 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:50.889344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb340bf1-52da-4f64-9943-18cdaa0b4c69 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:50.889684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb340bf1-52da-4f64-9943-18cdaa0b4c69 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:50.891320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 690/1369] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:50.896868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c8a3f-65e4-4789-8dc5-ad001326540e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:50.897317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c8a3f-65e4-4789-8dc5-ad001326540e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:50.897435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c8a3f-65e4-4789-8dc5-ad001326540e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:50.929941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:50.976275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a71c8a3f-65e4-4789-8dc5-ad001326540e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:50.977544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a71c8a3f-65e4-4789-8dc5-ad001326540e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:51.082327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 680/1370] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:41:51.422052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c913287-e866-450a-b798-b608bc5a9e86 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:51.422052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c913287-e866-450a-b798-b608bc5a9e86 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:51.422052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c913287-e866-450a-b798-b608bc5a9e86 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:51.422052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 691/1371] 10.222.0.20 () {54 vars in 963 bytes} [Tue Jul 12 03:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:51.427150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4316e-9ba3-4c49-b9ca-c40d499506bb None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:51.427383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4316e-9ba3-4c49-b9ca-c40d499506bb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:51.427817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4316e-9ba3-4c49-b9ca-c40d499506bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:51.504174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-79f4316e-9ba3-4c49-b9ca-c40d499506bb None None] MFA Rules not processed for user `cee6da907b614dc5b6ffb47e2860f459`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:51.505377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-79f4316e-9ba3-4c49-b9ca-c40d499506bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:51.555965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:51.600358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 681/1372] 10.222.0.20 () {58 vars in 1060 bytes} [Tue Jul 12 03:41:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:41:51.607793 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c913287-e866-450a-b798-b608bc5a9e86 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:51.609345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7c913287-e866-450a-b798-b608bc5a9e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:51.722456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c913287-e866-450a-b798-b608bc5a9e86 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:51.768225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:51.804282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885250b3-3f64-42fd-b8b7-ca18ae8a2bdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:51.805119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885250b3-3f64-42fd-b8b7-ca18ae8a2bdc service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:51.805501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885250b3-3f64-42fd-b8b7-ca18ae8a2bdc service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:51.805873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885250b3-3f64-42fd-b8b7-ca18ae8a2bdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:51.807840 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-885250b3-3f64-42fd-b8b7-ca18ae8a2bdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:51.923674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885250b3-3f64-42fd-b8b7-ca18ae8a2bdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:51.925480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885250b3-3f64-42fd-b8b7-ca18ae8a2bdc service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:51.945064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 692/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:52.048084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d78d4-bec3-416a-be97-87f75ca994d9 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:52.048567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d78d4-bec3-416a-be97-87f75ca994d9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:52.048811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d78d4-bec3-416a-be97-87f75ca994d9 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:52.050790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 682/1374] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:52.055597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eb750-bdfc-4f3d-9e16-d1d0080b86e1 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:52.055883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eb750-bdfc-4f3d-9e16-d1d0080b86e1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:52.056343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eb750-bdfc-4f3d-9e16-d1d0080b86e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:52.131043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-273eb750-bdfc-4f3d-9e16-d1d0080b86e1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:52.132468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-273eb750-bdfc-4f3d-9e16-d1d0080b86e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:52.154585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:52.248658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 693/1375] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:52.283623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d78d4-bec3-416a-be97-87f75ca994d9 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:52.285205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-613d78d4-bec3-416a-be97-87f75ca994d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:52.443436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d78d4-bec3-416a-be97-87f75ca994d9 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:52.497301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:52.562725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbfb2e-451b-4555-b7a6-08583d4774e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:52.563389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbfb2e-451b-4555-b7a6-08583d4774e7 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:52.563671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbfb2e-451b-4555-b7a6-08583d4774e7 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:52.563943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbfb2e-451b-4555-b7a6-08583d4774e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:52.565783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d3bbfb2e-451b-4555-b7a6-08583d4774e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:52.664473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbfb2e-451b-4555-b7a6-08583d4774e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:52.666133 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbfb2e-451b-4555-b7a6-08583d4774e7 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:52.685212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 683/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:54.745019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c7d6c-a58d-4fc9-bb52-59987b595afb None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:54.745579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c7d6c-a58d-4fc9-bb52-59987b595afb None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:54.745579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c7d6c-a58d-4fc9-bb52-59987b595afb None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:54.746992 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 694/1377] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:54.752415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32379ae8-baff-4d2b-88d5-bacc67dde113 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:54.752861 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32379ae8-baff-4d2b-88d5-bacc67dde113 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:54.752965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32379ae8-baff-4d2b-88d5-bacc67dde113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:54.839579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-32379ae8-baff-4d2b-88d5-bacc67dde113 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:54.841145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-32379ae8-baff-4d2b-88d5-bacc67dde113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:54.849478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:54.964359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 684/1378] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:55.008831 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c7d6c-a58d-4fc9-bb52-59987b595afb None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:55.010431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-af2c7d6c-a58d-4fc9-bb52-59987b595afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:55.171362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c7d6c-a58d-4fc9-bb52-59987b595afb None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:55.175240 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:55.297060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb82ef4-0c3a-4f85-9884-f11e33456a4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:55.297908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb82ef4-0c3a-4f85-9884-f11e33456a4f service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:55.298195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb82ef4-0c3a-4f85-9884-f11e33456a4f service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:55.298469 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb82ef4-0c3a-4f85-9884-f11e33456a4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:55.300813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3cb82ef4-0c3a-4f85-9884-f11e33456a4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:55.413169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb82ef4-0c3a-4f85-9884-f11e33456a4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:55.414919 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb82ef4-0c3a-4f85-9884-f11e33456a4f service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:55.438532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 695/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:41:57.506599 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16992c39-79df-4100-ba71-c6bd9bf49585 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:57.507193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16992c39-79df-4100-ba71-c6bd9bf49585 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:57.507193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16992c39-79df-4100-ba71-c6bd9bf49585 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:57.509205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 685/1380] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:41:57.514922 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b9361-eeaa-4200-8f7f-ee3de1e36b94 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:57.515564 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b9361-eeaa-4200-8f7f-ee3de1e36b94 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:57.515564 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b9361-eeaa-4200-8f7f-ee3de1e36b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:57.526037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:57.585308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-941b9361-eeaa-4200-8f7f-ee3de1e36b94 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:41:57.587213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-941b9361-eeaa-4200-8f7f-ee3de1e36b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:57.694156 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 696/1381] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:41:57.737581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16992c39-79df-4100-ba71-c6bd9bf49585 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:41:57.739364 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-16992c39-79df-4100-ba71-c6bd9bf49585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:57.890210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:57.903905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16992c39-79df-4100-ba71-c6bd9bf49585 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:41:58.027684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a411f-87c8-4ffa-8bbf-d846089a3f19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:41:58.028386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a411f-87c8-4ffa-8bbf-d846089a3f19 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:41:58.028670 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a411f-87c8-4ffa-8bbf-d846089a3f19 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:41:58.028939 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a411f-87c8-4ffa-8bbf-d846089a3f19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:41:58.030561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-822a411f-87c8-4ffa-8bbf-d846089a3f19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:41:58.124835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a411f-87c8-4ffa-8bbf-d846089a3f19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:41:58.126380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a411f-87c8-4ffa-8bbf-d846089a3f19 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:41:58.137963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:41:58.148175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 686/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:41:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:00.223137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902c7cb-da61-4bdf-a6a7-996518c28add None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:00.223851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902c7cb-da61-4bdf-a6a7-996518c28add None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:00.223851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902c7cb-da61-4bdf-a6a7-996518c28add None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:00.225068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 697/1383] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:00.231604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982448da-88b1-4899-8f56-e26b597ffd7b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:00.232134 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982448da-88b1-4899-8f56-e26b597ffd7b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:00.232439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982448da-88b1-4899-8f56-e26b597ffd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:00.309077 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-982448da-88b1-4899-8f56-e26b597ffd7b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:00.310449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-982448da-88b1-4899-8f56-e26b597ffd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:00.420591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 687/1384] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:00.464526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902c7cb-da61-4bdf-a6a7-996518c28add None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:00.466140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c902c7cb-da61-4bdf-a6a7-996518c28add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:00.560700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:00.629297 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902c7cb-da61-4bdf-a6a7-996518c28add None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:00.766315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1530d5-76b5-4aa0-aed4-50f3a47728f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:00.767026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1530d5-76b5-4aa0-aed4-50f3a47728f8 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:00.767311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1530d5-76b5-4aa0-aed4-50f3a47728f8 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:00.767588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1530d5-76b5-4aa0-aed4-50f3a47728f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:00.769460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3a1530d5-76b5-4aa0-aed4-50f3a47728f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:00.821836 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:00.867158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1530d5-76b5-4aa0-aed4-50f3a47728f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:00.868871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1530d5-76b5-4aa0-aed4-50f3a47728f8 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:00.888711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 698/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:03.030988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6c68-dc39-40da-9e00-8df7fd60e364 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:03.031613 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6c68-dc39-40da-9e00-8df7fd60e364 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:03.031613 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6c68-dc39-40da-9e00-8df7fd60e364 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:03.033107 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 688/1386] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:03.039845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126a795-c819-44b3-8264-5a423644c2ef None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:03.040396 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126a795-c819-44b3-8264-5a423644c2ef None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:03.040527 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126a795-c819-44b3-8264-5a423644c2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:03.109570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-4126a795-c819-44b3-8264-5a423644c2ef None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:03.110883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4126a795-c819-44b3-8264-5a423644c2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:03.213269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 699/1387] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:03.262209 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6c68-dc39-40da-9e00-8df7fd60e364 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:03.263839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a67e6c68-dc39-40da-9e00-8df7fd60e364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:03.345299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:03.418929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6c68-dc39-40da-9e00-8df7fd60e364 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:03.535408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ce9eb-0fa9-4393-bfc1-eb0f11de4e2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:03.536033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ce9eb-0fa9-4393-bfc1-eb0f11de4e2e service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:03.536317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ce9eb-0fa9-4393-bfc1-eb0f11de4e2e service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:03.536590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ce9eb-0fa9-4393-bfc1-eb0f11de4e2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:03.538291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0f4ce9eb-0fa9-4393-bfc1-eb0f11de4e2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:03.581075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:03.636882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ce9eb-0fa9-4393-bfc1-eb0f11de4e2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:03.638530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ce9eb-0fa9-4393-bfc1-eb0f11de4e2e service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:03.657098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 689/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:05.710699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebabfc-cf26-4284-970a-af0371459d8f None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:05.711279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebabfc-cf26-4284-970a-af0371459d8f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:05.711279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebabfc-cf26-4284-970a-af0371459d8f None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:05.712862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 700/1389] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:05.717990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e64ca-aa9a-46d5-a849-2def88b6b3e7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:05.718408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e64ca-aa9a-46d5-a849-2def88b6b3e7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:05.718513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e64ca-aa9a-46d5-a849-2def88b6b3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:05.785504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-303e64ca-aa9a-46d5-a849-2def88b6b3e7 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:05.786714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-303e64ca-aa9a-46d5-a849-2def88b6b3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:05.889894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 690/1390] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:05.933384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebabfc-cf26-4284-970a-af0371459d8f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:05.934889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e3ebabfc-cf26-4284-970a-af0371459d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:05.971043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:06.091328 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebabfc-cf26-4284-970a-af0371459d8f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:06.316618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:06.329035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a484f7-008c-43ea-b18b-8a5c8147c105 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:06.329677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a484f7-008c-43ea-b18b-8a5c8147c105 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:06.329953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a484f7-008c-43ea-b18b-8a5c8147c105 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:06.330218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a484f7-008c-43ea-b18b-8a5c8147c105 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:06.331850 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a9a484f7-008c-43ea-b18b-8a5c8147c105 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:06.421127 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a484f7-008c-43ea-b18b-8a5c8147c105 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:06.422726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a484f7-008c-43ea-b18b-8a5c8147c105 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:06.440916 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 701/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:06.984798 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76d91-a83c-4aa2-93b6-57c76bd20544 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:06.985426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76d91-a83c-4aa2-93b6-57c76bd20544 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:06.985568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76d91-a83c-4aa2-93b6-57c76bd20544 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:06.987643 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 691/1392] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:42:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:06.994634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33add8-b4cf-4d3f-b1f1-58c426a264b1 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:06.995120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33add8-b4cf-4d3f-b1f1-58c426a264b1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:06.995226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33add8-b4cf-4d3f-b1f1-58c426a264b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:07.062198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-ea33add8-b4cf-4d3f-b1f1-58c426a264b1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:07.063448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ea33add8-b4cf-4d3f-b1f1-58c426a264b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:07.127212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:07.168126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 702/1393] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:07.222913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90ba27-603c-494b-930d-f99ca58a2c05 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:07.223227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90ba27-603c-494b-930d-f99ca58a2c05 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:07.223505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90ba27-603c-494b-930d-f99ca58a2c05 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:07.224980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 692/1394] 10.222.0.20 () {54 vars in 962 bytes} [Tue Jul 12 03:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:07.230315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444e915-6619-4e51-9ff2-a945411ebb8c None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:07.230605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444e915-6619-4e51-9ff2-a945411ebb8c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:07.230888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444e915-6619-4e51-9ff2-a945411ebb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:07.305409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d444e915-6619-4e51-9ff2-a945411ebb8c None None] MFA Rules not processed for user `7297224dc73c4358b60f1b096c0fb5bf`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:07.306610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d444e915-6619-4e51-9ff2-a945411ebb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:07.392028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 703/1395] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Jul 12 03:42:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:42:07.395309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90ba27-603c-494b-930d-f99ca58a2c05 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:07.396877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0e90ba27-603c-494b-930d-f99ca58a2c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:07.411526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:07.504179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90ba27-603c-494b-930d-f99ca58a2c05 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:07.584524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea95794-aabb-4c48-901a-54a77af926ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7297224dc73c4358b60f1b096c0fb5bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:07.585197 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea95794-aabb-4c48-901a-54a77af926ac service glance] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:07.585482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea95794-aabb-4c48-901a-54a77af926ac service glance] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:07.585757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea95794-aabb-4c48-901a-54a77af926ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:07.587435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0ea95794-aabb-4c48-901a-54a77af926ac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:07.676388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:07.719785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea95794-aabb-4c48-901a-54a77af926ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:07.721287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea95794-aabb-4c48-901a-54a77af926ac service glance] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:07.739271 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 693/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:08.454413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0f874-97b8-4b36-a078-40f742300604 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:08.454920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0f874-97b8-4b36-a078-40f742300604 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:08.454920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0f874-97b8-4b36-a078-40f742300604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:08.469001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc97c2c-6c7a-4936-a3ae-f05effeaac0d None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:08.469539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc97c2c-6c7a-4936-a3ae-f05effeaac0d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:08.469622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc97c2c-6c7a-4936-a3ae-f05effeaac0d None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:08.470951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 694/1397] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:42:08.476224 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f890-dc39-472a-8fb9-5eb6ace859bb None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:08.476533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f890-dc39-472a-8fb9-5eb6ace859bb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:08.476806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1f890-dc39-472a-8fb9-5eb6ace859bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:08.522258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-09f0f874-97b8-4b36-a078-40f742300604 None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:08.523463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-09f0f874-97b8-4b36-a078-40f742300604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:08.532132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:08.543021 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-81f1f890-dc39-472a-8fb9-5eb6ace859bb None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:08.544246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-81f1f890-dc39-472a-8fb9-5eb6ace859bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:08.613859 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 704/1398] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:42:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:08.645202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 695/1399] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:08.656720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecd108-4f35-478d-9c6e-979152e9908f None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:08.657008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecd108-4f35-478d-9c6e-979152e9908f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:08.657284 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecd108-4f35-478d-9c6e-979152e9908f None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:08.658791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 705/1400] 10.222.0.20 () {54 vars in 942 bytes} [Tue Jul 12 03:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:08.666869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec768919-757b-4a2f-89ef-7320c53075ba None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:08.667162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec768919-757b-4a2f-89ef-7320c53075ba None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:08.667437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec768919-757b-4a2f-89ef-7320c53075ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:08.689741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecd108-4f35-478d-9c6e-979152e9908f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:08.691279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-50ecd108-4f35-478d-9c6e-979152e9908f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:08.693165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:08.746788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ec768919-757b-4a2f-89ef-7320c53075ba None None] MFA Rules not processed for user `0026b8f11158492b90da3395cc2c43c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:08.747958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ec768919-757b-4a2f-89ef-7320c53075ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:08.842015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecd108-4f35-478d-9c6e-979152e9908f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:08.845744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:08.854871 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 696/1401] 10.222.0.20 () {58 vars in 1039 bytes} [Tue Jul 12 03:42:08 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:08.865060 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec768919-757b-4a2f-89ef-7320c53075ba None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:08.874581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec768919-757b-4a2f-89ef-7320c53075ba None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:08.947470 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:08.963422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a66fb5-db4e-4fa2-905a-c2a38641f2ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:08.964184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a66fb5-db4e-4fa2-905a-c2a38641f2ce service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:08.964468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a66fb5-db4e-4fa2-905a-c2a38641f2ce service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:08.964743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a66fb5-db4e-4fa2-905a-c2a38641f2ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:08.966444 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-89a66fb5-db4e-4fa2-905a-c2a38641f2ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:08.983004 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd33210-6134-412a-9d72-efc55d706f19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:08.983645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd33210-6134-412a-9d72-efc55d706f19 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:08.983925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd33210-6134-412a-9d72-efc55d706f19 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:08.984232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd33210-6134-412a-9d72-efc55d706f19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:08.985896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8fd33210-6134-412a-9d72-efc55d706f19 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:09.057796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a66fb5-db4e-4fa2-905a-c2a38641f2ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:09.059426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a66fb5-db4e-4fa2-905a-c2a38641f2ce service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:09.075526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd33210-6134-412a-9d72-efc55d706f19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:09.077843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd33210-6134-412a-9d72-efc55d706f19 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:09.080573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 706/1402] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:42:09.088250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 697/1403] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:42:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 03:42:11.154291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37731a4-9db1-491c-a979-7c9dcd4c1529 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:11.154873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37731a4-9db1-491c-a979-7c9dcd4c1529 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:11.154873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37731a4-9db1-491c-a979-7c9dcd4c1529 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:11.162527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f0506-cc60-48e4-bf6d-458ffd1566b8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:11.163179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f0506-cc60-48e4-bf6d-458ffd1566b8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:11.163179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f0506-cc60-48e4-bf6d-458ffd1566b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:11.166614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 707/1404] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:42:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:11.250799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-180f0506-cc60-48e4-bf6d-458ffd1566b8 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:11.251930 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-180f0506-cc60-48e4-bf6d-458ffd1566b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:11.338084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:11.396727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 698/1405] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:11.441913 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37731a4-9db1-491c-a979-7c9dcd4c1529 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:11.443573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e37731a4-9db1-491c-a979-7c9dcd4c1529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:11.454545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:11.615784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37731a4-9db1-491c-a979-7c9dcd4c1529 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:11.754788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:11.819828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e0734-b6e3-48d2-b172-b3fd0b39f521 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:11.820671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e0734-b6e3-48d2-b172-b3fd0b39f521 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:11.822711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e0734-b6e3-48d2-b172-b3fd0b39f521 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:11.822711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e0734-b6e3-48d2-b172-b3fd0b39f521 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:11.822982 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f20e0734-b6e3-48d2-b172-b3fd0b39f521 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:11.920313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e0734-b6e3-48d2-b172-b3fd0b39f521 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:11.921960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e0734-b6e3-48d2-b172-b3fd0b39f521 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:11.953379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 708/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:14.014003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d6e42-85b8-405d-a730-3736698f52ba None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:14.014591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d6e42-85b8-405d-a730-3736698f52ba None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:14.014591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d6e42-85b8-405d-a730-3736698f52ba None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:14.015962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 699/1407] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:14.021394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2a673-d9d0-44d5-b8e8-de489effec43 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:14.021835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2a673-d9d0-44d5-b8e8-de489effec43 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:14.021972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2a673-d9d0-44d5-b8e8-de489effec43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:14.096824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-e7f2a673-d9d0-44d5-b8e8-de489effec43 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:14.098031 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e7f2a673-d9d0-44d5-b8e8-de489effec43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:14.152085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:14.219857 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 709/1408] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:14.257244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d6e42-85b8-405d-a730-3736698f52ba None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:14.259388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-878d6e42-85b8-405d-a730-3736698f52ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:14.425401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d6e42-85b8-405d-a730-3736698f52ba None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:14.468804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:14.554054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c59eb-1217-4487-bb72-3cf0328d63b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:14.554726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c59eb-1217-4487-bb72-3cf0328d63b3 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:14.555010 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c59eb-1217-4487-bb72-3cf0328d63b3 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:14.555285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c59eb-1217-4487-bb72-3cf0328d63b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:14.556996 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d45c59eb-1217-4487-bb72-3cf0328d63b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:14.660939 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c59eb-1217-4487-bb72-3cf0328d63b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:14.662585 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c59eb-1217-4487-bb72-3cf0328d63b3 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:14.684684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 700/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:24.210675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=23 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:24.679053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=79 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:30.825757 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bca75-d6e4-4acb-90db-ee214fcc7f7e None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:30.826356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bca75-d6e4-4acb-90db-ee214fcc7f7e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:30.826356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bca75-d6e4-4acb-90db-ee214fcc7f7e None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:30.827723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 710/1410] 10.222.0.20 () {54 vars in 918 bytes} [Tue Jul 12 03:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:30.834101 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e21df-8f00-420b-be53-173bb63aec73 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:30.834561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e21df-8f00-420b-be53-173bb63aec73 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:30.834671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e21df-8f00-420b-be53-173bb63aec73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:30.914846 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ae7e21df-8f00-420b-be53-173bb63aec73 None None] MFA Rules not processed for user `eded06390a2744c9aa52f77e3bfc2db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:30.916098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ae7e21df-8f00-420b-be53-173bb63aec73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:31.037144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 701/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Jul 12 03:42:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:31.059185 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b2fc4-579a-451c-8c18-e225a33062cc None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:31.059550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b2fc4-579a-451c-8c18-e225a33062cc None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:31.059834 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b2fc4-579a-451c-8c18-e225a33062cc None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:31.061965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 711/1412] 10.222.0.20 () {54 vars in 918 bytes} [Tue Jul 12 03:42:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:31.068172 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4146a19-be9d-448a-ab28-6440ec80b3a9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:31.068458 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4146a19-be9d-448a-ab28-6440ec80b3a9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:31.068730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4146a19-be9d-448a-ab28-6440ec80b3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:31.149131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:31.152561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d4146a19-be9d-448a-ab28-6440ec80b3a9 None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:31.154285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d4146a19-be9d-448a-ab28-6440ec80b3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:31.277371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 702/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Tue Jul 12 03:42:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:33.789508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df76de-3178-4dab-a63d-10b193799c93 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:33.790032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df76de-3178-4dab-a63d-10b193799c93 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:33.790032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df76de-3178-4dab-a63d-10b193799c93 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:33.792086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:42:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:33.801169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0567c58-4e63-4bc1-99cf-3cb2031ba2f7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:33.801596 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0567c58-4e63-4bc1-99cf-3cb2031ba2f7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:33.801741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0567c58-4e63-4bc1-99cf-3cb2031ba2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:33.875434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b0567c58-4e63-4bc1-99cf-3cb2031ba2f7 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:33.877082 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b0567c58-4e63-4bc1-99cf-3cb2031ba2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:33.934489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:33.978380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:34.022531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df76de-3178-4dab-a63d-10b193799c93 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:34.025214 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-65df76de-3178-4dab-a63d-10b193799c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:34.214301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df76de-3178-4dab-a63d-10b193799c93 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:34.308406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:34.353972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9cc13-2690-4f1e-8040-4dce49dcb855 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:34.354614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9cc13-2690-4f1e-8040-4dce49dcb855 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:34.354888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9cc13-2690-4f1e-8040-4dce49dcb855 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:34.355159 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9cc13-2690-4f1e-8040-4dce49dcb855 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:34.356928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2ce9cc13-2690-4f1e-8040-4dce49dcb855 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:34.462649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9cc13-2690-4f1e-8040-4dce49dcb855 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:34.464405 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9cc13-2690-4f1e-8040-4dce49dcb855 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:34.482777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 713/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:35.396671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfbcb3-2ce9-401d-87e4-4221e382e976 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:35.397259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfbcb3-2ce9-401d-87e4-4221e382e976 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:35.397259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfbcb3-2ce9-401d-87e4-4221e382e976 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:35.399307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 03:42:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:35.408245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306ae7d-6c7d-462f-b0fc-aee1d11f9d26 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:35.408653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306ae7d-6c7d-462f-b0fc-aee1d11f9d26 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:35.408863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306ae7d-6c7d-462f-b0fc-aee1d11f9d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:35.486121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-c306ae7d-6c7d-462f-b0fc-aee1d11f9d26 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:35.487376 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c306ae7d-6c7d-462f-b0fc-aee1d11f9d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:35.517107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:35.599419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 03:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:35.762539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfbcb3-2ce9-401d-87e4-4221e382e976 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:35.765510 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9dbfbcb3-2ce9-401d-87e4-4221e382e976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:35.923725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfbcb3-2ce9-401d-87e4-4221e382e976 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:35.968209 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:36.054190 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfb61a-29b8-4196-8e5b-58b079621872 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:36.054771 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfb61a-29b8-4196-8e5b-58b079621872 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:36.055025 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfb61a-29b8-4196-8e5b-58b079621872 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:36.055291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfb61a-29b8-4196-8e5b-58b079621872 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:36.057003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-96dfb61a-29b8-4196-8e5b-58b079621872 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:36.160042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfb61a-29b8-4196-8e5b-58b079621872 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:36.161678 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfb61a-29b8-4196-8e5b-58b079621872 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:36.191400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 705/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:37.090033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9f8f9-ee8c-4b46-bc3a-249c6412e212 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:37.090897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9f8f9-ee8c-4b46-bc3a-249c6412e212 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:37.090897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9f8f9-ee8c-4b46-bc3a-249c6412e212 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:37.093043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 03:42:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:37.099162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4858c0-0dd8-4c66-b03c-68d3ae473c4b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:37.099634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4858c0-0dd8-4c66-b03c-68d3ae473c4b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:37.099804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4858c0-0dd8-4c66-b03c-68d3ae473c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:37.166198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:37.190288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ef4858c0-0dd8-4c66-b03c-68d3ae473c4b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:37.191540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ef4858c0-0dd8-4c66-b03c-68d3ae473c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:37.304316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 03:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:37.458378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9f8f9-ee8c-4b46-bc3a-249c6412e212 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:37.460967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7ae9f8f9-ee8c-4b46-bc3a-249c6412e212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:37.629098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9f8f9-ee8c-4b46-bc3a-249c6412e212 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:37.641953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:37.752642 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ddc60-93e4-46f5-89db-e9c1e2cb8ede service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:37.753387 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ddc60-93e4-46f5-89db-e9c1e2cb8ede service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:37.753675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ddc60-93e4-46f5-89db-e9c1e2cb8ede service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:37.753945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ddc60-93e4-46f5-89db-e9c1e2cb8ede service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:37.755585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-907ddc60-93e4-46f5-89db-e9c1e2cb8ede service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:37.852617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ddc60-93e4-46f5-89db-e9c1e2cb8ede service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:37.854246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ddc60-93e4-46f5-89db-e9c1e2cb8ede service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:37.873217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 716/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:38.734827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d1144-b7a3-4882-88c9-5dc57c5cdd04 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:38.735407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d1144-b7a3-4882-88c9-5dc57c5cdd04 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:38.735407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d1144-b7a3-4882-88c9-5dc57c5cdd04 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:38.736803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 03:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:38.743472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308260ae-3bb3-485c-84ca-ed7182fbf174 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:38.743943 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308260ae-3bb3-485c-84ca-ed7182fbf174 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:38.744112 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308260ae-3bb3-485c-84ca-ed7182fbf174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:38.773123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:38.840041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-308260ae-3bb3-485c-84ca-ed7182fbf174 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:38.841253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-308260ae-3bb3-485c-84ca-ed7182fbf174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:38.951068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 03:42:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:39.102807 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d1144-b7a3-4882-88c9-5dc57c5cdd04 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:39.104452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-da4d1144-b7a3-4882-88c9-5dc57c5cdd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:39.227306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:39.267697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d1144-b7a3-4882-88c9-5dc57c5cdd04 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:39.400256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913e0c8-6f1d-4c2e-ba94-12d269dc4c7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:39.400812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913e0c8-6f1d-4c2e-ba94-12d269dc4c7b service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:39.401093 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913e0c8-6f1d-4c2e-ba94-12d269dc4c7b service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:39.401367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913e0c8-6f1d-4c2e-ba94-12d269dc4c7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:39.403026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0913e0c8-6f1d-4c2e-ba94-12d269dc4c7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:39.489716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:39.504493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913e0c8-6f1d-4c2e-ba94-12d269dc4c7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:39.506124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913e0c8-6f1d-4c2e-ba94-12d269dc4c7b service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:39.528496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 708/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:40.384772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3890f-173a-4b78-8192-19d47485db0c None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:40.385604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3890f-173a-4b78-8192-19d47485db0c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:40.385604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3890f-173a-4b78-8192-19d47485db0c None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:40.387578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 03:42:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:40.394873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75fdd2-30b9-4e12-b143-58d3ac01c6c8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:40.395309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75fdd2-30b9-4e12-b143-58d3ac01c6c8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:40.395450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75fdd2-30b9-4e12-b143-58d3ac01c6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:40.469600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-bb75fdd2-30b9-4e12-b143-58d3ac01c6c8 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:40.470790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bb75fdd2-30b9-4e12-b143-58d3ac01c6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:40.570192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 03:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:40.728876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3890f-173a-4b78-8192-19d47485db0c None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:40.730446 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f8a3890f-173a-4b78-8192-19d47485db0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:40.828905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:40.897957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3890f-173a-4b78-8192-19d47485db0c None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:41.027206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a2a86-5f88-4ebb-aa0c-0866c20c7e32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:41.027865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a2a86-5f88-4ebb-aa0c-0866c20c7e32 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:41.028183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a2a86-5f88-4ebb-aa0c-0866c20c7e32 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:41.028466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a2a86-5f88-4ebb-aa0c-0866c20c7e32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:41.030106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1a0a2a86-5f88-4ebb-aa0c-0866c20c7e32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:41.087417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:41.137561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a2a86-5f88-4ebb-aa0c-0866c20c7e32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:41.139196 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a2a86-5f88-4ebb-aa0c-0866c20c7e32 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:41.158930 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 719/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:41.960844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b8b14-8dd5-493a-b761-ee6a0e308c2b None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:41.961413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b8b14-8dd5-493a-b761-ee6a0e308c2b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:41.961413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b8b14-8dd5-493a-b761-ee6a0e308c2b None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:41.962977 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 03:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:41.970463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31318d-5906-403d-9328-888482c5bf33 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:41.970881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31318d-5906-403d-9328-888482c5bf33 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:41.971023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31318d-5906-403d-9328-888482c5bf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:42.052220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-fe31318d-5906-403d-9328-888482c5bf33 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:42.053422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-fe31318d-5906-403d-9328-888482c5bf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:42.152677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 03:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:42.305436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b8b14-8dd5-493a-b761-ee6a0e308c2b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:42.307875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ac4b8b14-8dd5-493a-b761-ee6a0e308c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:42.370467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:42.482829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b8b14-8dd5-493a-b761-ee6a0e308c2b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:42.611556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343253dd-547d-4dca-8297-7322ac8cb668 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:42.612407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343253dd-547d-4dca-8297-7322ac8cb668 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:42.612705 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343253dd-547d-4dca-8297-7322ac8cb668 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:42.612966 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343253dd-547d-4dca-8297-7322ac8cb668 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:42.614602 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-343253dd-547d-4dca-8297-7322ac8cb668 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:42.624491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:42.720216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343253dd-547d-4dca-8297-7322ac8cb668 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:42.721901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343253dd-547d-4dca-8297-7322ac8cb668 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:42.742623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 711/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:43.548311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792c63e-0975-47db-af41-57b05fb0b427 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:43.548939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792c63e-0975-47db-af41-57b05fb0b427 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:43.548939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792c63e-0975-47db-af41-57b05fb0b427 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:43.550455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 03:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:43.556758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56214328-10e6-4c8f-bcd1-175c6a963f90 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:43.557396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56214328-10e6-4c8f-bcd1-175c6a963f90 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:43.557396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56214328-10e6-4c8f-bcd1-175c6a963f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:43.644700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-56214328-10e6-4c8f-bcd1-175c6a963f90 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:43.645904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-56214328-10e6-4c8f-bcd1-175c6a963f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:43.725042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:43.754994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 03:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:43.907262 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792c63e-0975-47db-af41-57b05fb0b427 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:43.908845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a792c63e-0975-47db-af41-57b05fb0b427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:43.928290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:44.066733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792c63e-0975-47db-af41-57b05fb0b427 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:44.166290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:44.194891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade99f0b-a99d-4e4b-9fe5-d169f4274cb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:44.195554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade99f0b-a99d-4e4b-9fe5-d169f4274cb9 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:44.195842 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade99f0b-a99d-4e4b-9fe5-d169f4274cb9 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:44.196171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade99f0b-a99d-4e4b-9fe5-d169f4274cb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:44.197969 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ade99f0b-a99d-4e4b-9fe5-d169f4274cb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:44.292434 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade99f0b-a99d-4e4b-9fe5-d169f4274cb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:44.294058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade99f0b-a99d-4e4b-9fe5-d169f4274cb9 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:44.312426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 722/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:45.121471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595023d6-4e52-4c65-8427-cfb829c4ed24 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:45.122080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595023d6-4e52-4c65-8427-cfb829c4ed24 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:45.122080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595023d6-4e52-4c65-8427-cfb829c4ed24 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:45.123399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 03:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:45.130140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c3ad7-8207-4d1f-b87a-aab61aa62aaa None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:45.130570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c3ad7-8207-4d1f-b87a-aab61aa62aaa None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:45.130695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c3ad7-8207-4d1f-b87a-aab61aa62aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:45.209503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a39c3ad7-8207-4d1f-b87a-aab61aa62aaa None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:45.210717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a39c3ad7-8207-4d1f-b87a-aab61aa62aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:45.252786 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:45.319627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 03:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:45.472623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595023d6-4e52-4c65-8427-cfb829c4ed24 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:45.475006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-595023d6-4e52-4c65-8427-cfb829c4ed24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:45.654141 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595023d6-4e52-4c65-8427-cfb829c4ed24 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:45.712054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:45.776069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9affdf-8587-4b89-b511-29a66185eda3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:45.777014 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9affdf-8587-4b89-b511-29a66185eda3 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:45.777417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9affdf-8587-4b89-b511-29a66185eda3 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:45.777787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9affdf-8587-4b89-b511-29a66185eda3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:45.780526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7b9affdf-8587-4b89-b511-29a66185eda3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:45.901792 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9affdf-8587-4b89-b511-29a66185eda3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:45.903445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9affdf-8587-4b89-b511-29a66185eda3 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:45.921588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 714/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:46.719278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd905-fc7c-47bf-86d3-0506f5ff2696 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:46.719889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd905-fc7c-47bf-86d3-0506f5ff2696 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:46.719889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd905-fc7c-47bf-86d3-0506f5ff2696 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:46.721251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 03:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:46.726367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb489ad-e43f-4a1f-9653-3079455742b2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:46.726880 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb489ad-e43f-4a1f-9653-3079455742b2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:46.726985 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb489ad-e43f-4a1f-9653-3079455742b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:46.814969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ebb489ad-e43f-4a1f-9653-3079455742b2 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:46.816304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ebb489ad-e43f-4a1f-9653-3079455742b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:46.818400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:46.920459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 03:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:47.072070 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dd905-fc7c-47bf-86d3-0506f5ff2696 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:47.074629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3a9dd905-fc7c-47bf-86d3-0506f5ff2696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:47.225336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dd905-fc7c-47bf-86d3-0506f5ff2696 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:47.252150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:47.349229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b91b6b-b381-4e34-bae2-b5fa8d4cd4b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:47.349876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b91b6b-b381-4e34-bae2-b5fa8d4cd4b0 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:47.350161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b91b6b-b381-4e34-bae2-b5fa8d4cd4b0 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:47.350437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b91b6b-b381-4e34-bae2-b5fa8d4cd4b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:47.352140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b5b91b6b-b381-4e34-bae2-b5fa8d4cd4b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:47.446980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b91b6b-b381-4e34-bae2-b5fa8d4cd4b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:47.448634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b91b6b-b381-4e34-bae2-b5fa8d4cd4b0 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:47.467586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 725/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:49.371132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c3f52-2794-433d-9689-2d24f3bbbfd3 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:49.372071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c3f52-2794-433d-9689-2d24f3bbbfd3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:49.372071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c3f52-2794-433d-9689-2d24f3bbbfd3 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:49.374262 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:42:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:49.383408 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1766a4-2dac-4f01-85b7-2ddab4778f75 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:49.383852 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1766a4-2dac-4f01-85b7-2ddab4778f75 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:49.384363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1766a4-2dac-4f01-85b7-2ddab4778f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:49.431801 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:49.466349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-6c1766a4-2dac-4f01-85b7-2ddab4778f75 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:49.467628 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6c1766a4-2dac-4f01-85b7-2ddab4778f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:49.566463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:49.608205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c3f52-2794-433d-9689-2d24f3bbbfd3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:49.609853 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-831c3f52-2794-433d-9689-2d24f3bbbfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:49.722956 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c3f52-2794-433d-9689-2d24f3bbbfd3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:49.756133 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:49.798997 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190443fa-398e-4bf7-b27f-919bbe93eca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:49.799648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190443fa-398e-4bf7-b27f-919bbe93eca0 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:49.799944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190443fa-398e-4bf7-b27f-919bbe93eca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:49.800249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190443fa-398e-4bf7-b27f-919bbe93eca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:49.801890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-190443fa-398e-4bf7-b27f-919bbe93eca0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:49.919352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190443fa-398e-4bf7-b27f-919bbe93eca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:49.920867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190443fa-398e-4bf7-b27f-919bbe93eca0 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:49.939630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 717/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:42:52.173588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8136a32-a384-4949-a1f7-e7dc70a86e5d None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:52.174145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8136a32-a384-4949-a1f7-e7dc70a86e5d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:52.174145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8136a32-a384-4949-a1f7-e7dc70a86e5d None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:52.175872 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:42:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:52.181477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2dbba-3c6f-497a-b164-ce988811d96a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:52.181909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2dbba-3c6f-497a-b164-ce988811d96a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:52.182061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2dbba-3c6f-497a-b164-ce988811d96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:52.248657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6cc2dbba-3c6f-497a-b164-ce988811d96a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:52.249861 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6cc2dbba-3c6f-497a-b164-ce988811d96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:52.264039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:52.354405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:52.392730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8136a32-a384-4949-a1f7-e7dc70a86e5d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:52.394462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d8136a32-a384-4949-a1f7-e7dc70a86e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:52.500862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8136a32-a384-4949-a1f7-e7dc70a86e5d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:52.519089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:52.585179 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6c136-44dd-4774-a0a5-d444de847231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:52.585822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6c136-44dd-4774-a0a5-d444de847231 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:52.586097 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6c136-44dd-4774-a0a5-d444de847231 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:52.586380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6c136-44dd-4774-a0a5-d444de847231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:52.587991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-cad6c136-44dd-4774-a0a5-d444de847231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:52.703947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6c136-44dd-4774-a0a5-d444de847231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:52.705485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6c136-44dd-4774-a0a5-d444de847231 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:52.724201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 728/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:55.048083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a98feb-2d7f-4338-bc9e-07d606df17f9 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:55.048748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a98feb-2d7f-4338-bc9e-07d606df17f9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:55.048748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a98feb-2d7f-4338-bc9e-07d606df17f9 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:55.049928 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:55.055292 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95984657-2ae8-4432-a6e7-f908fd90ecbc None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:55.055736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95984657-2ae8-4432-a6e7-f908fd90ecbc None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:55.055831 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95984657-2ae8-4432-a6e7-f908fd90ecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:55.103810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:55.121805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-95984657-2ae8-4432-a6e7-f908fd90ecbc None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:55.123008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-95984657-2ae8-4432-a6e7-f908fd90ecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:55.223513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:55.265481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a98feb-2d7f-4338-bc9e-07d606df17f9 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:55.267847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-57a98feb-2d7f-4338-bc9e-07d606df17f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:55.383159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a98feb-2d7f-4338-bc9e-07d606df17f9 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:55.426104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:55.462795 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76533159-7915-46f0-b97f-21260ebfe861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:55.463448 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76533159-7915-46f0-b97f-21260ebfe861 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:55.463731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76533159-7915-46f0-b97f-21260ebfe861 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:55.464040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76533159-7915-46f0-b97f-21260ebfe861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:55.465688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-76533159-7915-46f0-b97f-21260ebfe861 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:55.579120 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76533159-7915-46f0-b97f-21260ebfe861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:55.580670 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76533159-7915-46f0-b97f-21260ebfe861 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:55.599231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 720/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:42:57.744900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea0e82-a77b-48b5-9b48-7de31a8521b1 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:57.745511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea0e82-a77b-48b5-9b48-7de31a8521b1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:57.745511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea0e82-a77b-48b5-9b48-7de31a8521b1 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:57.747685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:42:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:42:57.756929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929b8a6-e86e-4b5c-baf5-a806ecef7ef9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:57.757352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929b8a6-e86e-4b5c-baf5-a806ecef7ef9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:57.757564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929b8a6-e86e-4b5c-baf5-a806ecef7ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:57.830129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0929b8a6-e86e-4b5c-baf5-a806ecef7ef9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:42:57.831415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0929b8a6-e86e-4b5c-baf5-a806ecef7ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:57.854206 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:57.945119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:42:57.998439 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea0e82-a77b-48b5-9b48-7de31a8521b1 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:42:58.000791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-79ea0e82-a77b-48b5-9b48-7de31a8521b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:58.121261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea0e82-a77b-48b5-9b48-7de31a8521b1 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:42:58.145227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:42:58.202083 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f653137-d495-4ff3-be90-cdbb32a5b4ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:42:58.202726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f653137-d495-4ff3-be90-cdbb32a5b4ee service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:42:58.203018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f653137-d495-4ff3-be90-cdbb32a5b4ee service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:42:58.203287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f653137-d495-4ff3-be90-cdbb32a5b4ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:42:58.204959 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6f653137-d495-4ff3-be90-cdbb32a5b4ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:42:58.327516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f653137-d495-4ff3-be90-cdbb32a5b4ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:42:58.329010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f653137-d495-4ff3-be90-cdbb32a5b4ee service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:42:58.348432 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 731/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:43:00.414982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ce55e-83c6-4816-8688-f172461c27d8 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:00.415593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ce55e-83c6-4816-8688-f172461c27d8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:00.415593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ce55e-83c6-4816-8688-f172461c27d8 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:00.416967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:00.423712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e78ad5-70e6-416e-a4b7-63ef59c5f968 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:00.424225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e78ad5-70e6-416e-a4b7-63ef59c5f968 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:00.424333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e78ad5-70e6-416e-a4b7-63ef59c5f968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:00.492804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:00.496201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-b0e78ad5-70e6-416e-a4b7-63ef59c5f968 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:00.497920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b0e78ad5-70e6-416e-a4b7-63ef59c5f968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:00.602486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:00.644513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ce55e-83c6-4816-8688-f172461c27d8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:00.646607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-978ce55e-83c6-4816-8688-f172461c27d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:00.757832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ce55e-83c6-4816-8688-f172461c27d8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:00.813462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:00.839361 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637e8b5-dc6a-4af7-b52c-1915060203d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:00.840029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637e8b5-dc6a-4af7-b52c-1915060203d2 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:00.840332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637e8b5-dc6a-4af7-b52c-1915060203d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:00.840628 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637e8b5-dc6a-4af7-b52c-1915060203d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:00.842244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5637e8b5-dc6a-4af7-b52c-1915060203d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:00.976373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637e8b5-dc6a-4af7-b52c-1915060203d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:00.977834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637e8b5-dc6a-4af7-b52c-1915060203d2 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:00.996840 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 723/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:43:03.052121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c60f62-edbe-40d6-8086-32fb831d0355 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:03.052711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c60f62-edbe-40d6-8086-32fb831d0355 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:03.052711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c60f62-edbe-40d6-8086-32fb831d0355 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:03.054077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:03.060524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5fc54-4351-4629-88ed-6ec443c1a61e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:03.061247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5fc54-4351-4629-88ed-6ec443c1a61e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:03.061436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5fc54-4351-4629-88ed-6ec443c1a61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:03.132464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-03b5fc54-4351-4629-88ed-6ec443c1a61e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:03.133681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-03b5fc54-4351-4629-88ed-6ec443c1a61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:03.169008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:03.251898 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:03.292659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c60f62-edbe-40d6-8086-32fb831d0355 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:03.294192 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-75c60f62-edbe-40d6-8086-32fb831d0355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:03.403997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c60f62-edbe-40d6-8086-32fb831d0355 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:03.432372 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:03.485157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cf78c-dd02-45b8-8068-d6ccded5a309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:03.485811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cf78c-dd02-45b8-8068-d6ccded5a309 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:03.486098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cf78c-dd02-45b8-8068-d6ccded5a309 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:03.486379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cf78c-dd02-45b8-8068-d6ccded5a309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:03.488071 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ad3cf78c-dd02-45b8-8068-d6ccded5a309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:03.600528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cf78c-dd02-45b8-8068-d6ccded5a309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:03.601996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cf78c-dd02-45b8-8068-d6ccded5a309 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:03.620787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 734/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:43:05.683704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1149d-4f21-4b66-acaa-39f3f0e80823 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:05.684711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1149d-4f21-4b66-acaa-39f3f0e80823 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:05.684711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1149d-4f21-4b66-acaa-39f3f0e80823 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:05.687630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:43:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:05.696411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b5392-1d65-4fb8-82fd-ef126cc5ed79 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:05.697192 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b5392-1d65-4fb8-82fd-ef126cc5ed79 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:05.697293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b5392-1d65-4fb8-82fd-ef126cc5ed79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:05.767400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2e2b5392-1d65-4fb8-82fd-ef126cc5ed79 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:05.768644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2e2b5392-1d65-4fb8-82fd-ef126cc5ed79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:05.777190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:05.873861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:05.919592 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1149d-4f21-4b66-acaa-39f3f0e80823 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:05.921174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3bd1149d-4f21-4b66-acaa-39f3f0e80823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:06.033128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1149d-4f21-4b66-acaa-39f3f0e80823 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:06.093855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:06.113484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4e027-4ef9-4a74-b8e2-01f061390d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:06.114139 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4e027-4ef9-4a74-b8e2-01f061390d34 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:06.114429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4e027-4ef9-4a74-b8e2-01f061390d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:06.114702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4e027-4ef9-4a74-b8e2-01f061390d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:06.116408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8cc4e027-4ef9-4a74-b8e2-01f061390d34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:06.238305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4e027-4ef9-4a74-b8e2-01f061390d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:06.239689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4e027-4ef9-4a74-b8e2-01f061390d34 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:06.259091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 726/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:43:08.335219 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75296011-1434-43d2-b877-995fd80def16 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:08.335783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75296011-1434-43d2-b877-995fd80def16 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:08.335783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75296011-1434-43d2-b877-995fd80def16 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:08.337189 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:08.343740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84503f2-4ce8-47c8-bc92-82b948127281 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:08.344553 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84503f2-4ce8-47c8-bc92-82b948127281 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:08.344754 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84503f2-4ce8-47c8-bc92-82b948127281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:08.413090 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c84503f2-4ce8-47c8-bc92-82b948127281 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:08.414302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c84503f2-4ce8-47c8-bc92-82b948127281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:08.462169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:08.526475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:43:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:08.563900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75296011-1434-43d2-b877-995fd80def16 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:08.565587 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-75296011-1434-43d2-b877-995fd80def16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:08.683477 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75296011-1434-43d2-b877-995fd80def16 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:08.723062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:08.760763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d31274-2031-49fa-8435-d2ae37c36391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:08.761420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d31274-2031-49fa-8435-d2ae37c36391 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:08.761704 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d31274-2031-49fa-8435-d2ae37c36391 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:08.761972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d31274-2031-49fa-8435-d2ae37c36391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:08.763639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c5d31274-2031-49fa-8435-d2ae37c36391 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:08.875445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d31274-2031-49fa-8435-d2ae37c36391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:08.876922 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d31274-2031-49fa-8435-d2ae37c36391 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:08.896936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 737/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:43:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:43:10.910499 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f9a31-660b-4c5c-af73-6e3b1d3460a7 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:10.911113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f9a31-660b-4c5c-af73-6e3b1d3460a7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:10.911113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f9a31-660b-4c5c-af73-6e3b1d3460a7 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:10.912433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:10.917731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598388ff-606d-491d-bd8c-3929dee49176 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:10.918174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598388ff-606d-491d-bd8c-3929dee49176 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:10.918268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598388ff-606d-491d-bd8c-3929dee49176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:10.995689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-598388ff-606d-491d-bd8c-3929dee49176 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:10.996980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-598388ff-606d-491d-bd8c-3929dee49176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:11.016853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:11.099069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:11.135274 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f9a31-660b-4c5c-af73-6e3b1d3460a7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:11.136878 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0c2f9a31-660b-4c5c-af73-6e3b1d3460a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:11.247121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f9a31-660b-4c5c-af73-6e3b1d3460a7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:11.315908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:11.324090 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e728944-e95f-4e0b-b279-f55c31ded062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:11.324752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e728944-e95f-4e0b-b279-f55c31ded062 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:11.325037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e728944-e95f-4e0b-b279-f55c31ded062 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:11.325312 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e728944-e95f-4e0b-b279-f55c31ded062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:11.326967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6e728944-e95f-4e0b-b279-f55c31ded062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:11.438218 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e728944-e95f-4e0b-b279-f55c31ded062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:11.439661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e728944-e95f-4e0b-b279-f55c31ded062 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:11.461947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 729/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:43:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:43:13.488276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0a5a2-63eb-4d22-9a42-21483f97203d None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:13.489571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0a5a2-63eb-4d22-9a42-21483f97203d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:13.489571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0a5a2-63eb-4d22-9a42-21483f97203d None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:13.491549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:43:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:13.498422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17642003-2b16-48a1-8949-143f5ea98411 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:13.499234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17642003-2b16-48a1-8949-143f5ea98411 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:13.499412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17642003-2b16-48a1-8949-143f5ea98411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:13.580832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-17642003-2b16-48a1-8949-143f5ea98411 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:13.582084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-17642003-2b16-48a1-8949-143f5ea98411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:13.651859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:13.702021 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:43:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:13.740808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0a5a2-63eb-4d22-9a42-21483f97203d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:13.743162 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-abc0a5a2-63eb-4d22-9a42-21483f97203d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:13.867526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0a5a2-63eb-4d22-9a42-21483f97203d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:13.924923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:13.957041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c2767-c2ea-43c1-a6a6-64535d7f7b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:13.957829 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c2767-c2ea-43c1-a6a6-64535d7f7b4b service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:13.958111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c2767-c2ea-43c1-a6a6-64535d7f7b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:13.958390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c2767-c2ea-43c1-a6a6-64535d7f7b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:13.960033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-365c2767-c2ea-43c1-a6a6-64535d7f7b4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:14.080295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c2767-c2ea-43c1-a6a6-64535d7f7b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:14.081766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c2767-c2ea-43c1-a6a6-64535d7f7b4b service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:14.101374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 740/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:43:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:43:23.700198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=22 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:24.098813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=66 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:34.466716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57537c-c439-4406-8a69-40e103a7c738 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:34.467579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57537c-c439-4406-8a69-40e103a7c738 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:34.467579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57537c-c439-4406-8a69-40e103a7c738 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:34.469844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 731/1471] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:43:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:34.475303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64580d-61d4-4b34-84c1-c86ebd59d502 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:34.475890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64580d-61d4-4b34-84c1-c86ebd59d502 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:34.475890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64580d-61d4-4b34-84c1-c86ebd59d502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:34.549172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-0e64580d-61d4-4b34-84c1-c86ebd59d502 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:34.550341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0e64580d-61d4-4b34-84c1-c86ebd59d502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:34.658044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 741/1472] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:34.755054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167c96e-9c97-4013-aa65-e34c7bb307cf None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:34.755460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167c96e-9c97-4013-aa65-e34c7bb307cf None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:34.755858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167c96e-9c97-4013-aa65-e34c7bb307cf None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:34.757802 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 732/1473] 10.222.0.20 () {54 vars in 962 bytes} [Tue Jul 12 03:43:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:34.763322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57295c89-8f7f-47d3-8e0e-8fdb73335764 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:34.763617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57295c89-8f7f-47d3-8e0e-8fdb73335764 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:34.763892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57295c89-8f7f-47d3-8e0e-8fdb73335764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:34.853646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-57295c89-8f7f-47d3-8e0e-8fdb73335764 None None] MFA Rules not processed for user `7297224dc73c4358b60f1b096c0fb5bf`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:34.854895 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-57295c89-8f7f-47d3-8e0e-8fdb73335764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:34.878591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:34.940414 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 742/1474] 10.222.0.20 () {58 vars in 1059 bytes} [Tue Jul 12 03:43:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:34.948129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167c96e-9c97-4013-aa65-e34c7bb307cf None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:34.949660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f167c96e-9c97-4013-aa65-e34c7bb307cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:35.057243 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167c96e-9c97-4013-aa65-e34c7bb307cf None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:35.131678 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925b52e-1752-4294-9003-a7ff51af3e42 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7297224dc73c4358b60f1b096c0fb5bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:35.132355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925b52e-1752-4294-9003-a7ff51af3e42 service glance] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:35.132635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925b52e-1752-4294-9003-a7ff51af3e42 service glance] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:35.132907 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925b52e-1752-4294-9003-a7ff51af3e42 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:35.134536 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4925b52e-1752-4294-9003-a7ff51af3e42 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:35.192029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:35.250526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925b52e-1752-4294-9003-a7ff51af3e42 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:35.251963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925b52e-1752-4294-9003-a7ff51af3e42 service glance] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:35.271479 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 733/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:43:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:43:37.405967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eada8e1-98b8-4a40-b200-164a919be355 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:37.406891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eada8e1-98b8-4a40-b200-164a919be355 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:37.406891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eada8e1-98b8-4a40-b200-164a919be355 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:37.409117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 743/1476] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:43:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:37.418227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473ad62-579c-4329-b997-0a21a19530ac None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:37.418676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473ad62-579c-4329-b997-0a21a19530ac None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:37.418822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473ad62-579c-4329-b997-0a21a19530ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:37.490507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8473ad62-579c-4329-b997-0a21a19530ac None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:37.491748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8473ad62-579c-4329-b997-0a21a19530ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:37.608666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 734/1477] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:37.632621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eada8e1-98b8-4a40-b200-164a919be355 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:37.634212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2eada8e1-98b8-4a40-b200-164a919be355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:37.785567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eada8e1-98b8-4a40-b200-164a919be355 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:37.837179 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:37.885728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4c99b-8ea7-423b-aabd-d571d5353230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:37.886755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4c99b-8ea7-423b-aabd-d571d5353230 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:37.887065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4c99b-8ea7-423b-aabd-d571d5353230 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:37.887336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4c99b-8ea7-423b-aabd-d571d5353230 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:37.888108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4c99b-8ea7-423b-aabd-d571d5353230 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:37.889879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: WARNING py.warnings [None req-91c4c99b-8ea7-423b-aabd-d571d5353230 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:43:37.889879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: warnings.warn(msg) Jul 12 03:43:37.889879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: Jul 12 03:43:37.891032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4c99b-8ea7-423b-aabd-d571d5353230 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:37.911169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 744/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Tue Jul 12 03:43:37 2022] GET /identity/v3/projects => generated 2553 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:43:39.951881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee74c63-5860-419e-99ef-0c2d30d93099 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:39.952480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee74c63-5860-419e-99ef-0c2d30d93099 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:39.952480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee74c63-5860-419e-99ef-0c2d30d93099 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:39.953867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 735/1479] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:39.959814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffa7f2-5f9e-4d4e-8913-095fe84d884e None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:39.960310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffa7f2-5f9e-4d4e-8913-095fe84d884e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:39.960423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cffa7f2-5f9e-4d4e-8913-095fe84d884e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:40.028578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-3cffa7f2-5f9e-4d4e-8913-095fe84d884e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:40.029812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3cffa7f2-5f9e-4d4e-8913-095fe84d884e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:40.136780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 745/1480] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:40.182465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee74c63-5860-419e-99ef-0c2d30d93099 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:40.184145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bee74c63-5860-419e-99ef-0c2d30d93099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:40.197986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:40.347173 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee74c63-5860-419e-99ef-0c2d30d93099 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:40.437855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:40.475002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211962dc-0465-4f21-ab52-f583d921e8d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:40.476116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211962dc-0465-4f21-ab52-f583d921e8d1 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:40.476445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211962dc-0465-4f21-ab52-f583d921e8d1 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:40.476720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211962dc-0465-4f21-ab52-f583d921e8d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:40.478406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-211962dc-0465-4f21-ab52-f583d921e8d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:40.572766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211962dc-0465-4f21-ab52-f583d921e8d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:40.574404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211962dc-0465-4f21-ab52-f583d921e8d1 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:40.594040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 736/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:43:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:43:42.822470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179e670-9c9a-4cdb-ad27-3c15f7d9a6d0 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:42.823080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179e670-9c9a-4cdb-ad27-3c15f7d9a6d0 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:42.823080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179e670-9c9a-4cdb-ad27-3c15f7d9a6d0 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:42.824445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 746/1482] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:42.830800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419e33a-8d72-4b47-91ac-ab51c56d78d8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:42.831558 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419e33a-8d72-4b47-91ac-ab51c56d78d8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:42.831827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419e33a-8d72-4b47-91ac-ab51c56d78d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:42.906700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b419e33a-8d72-4b47-91ac-ab51c56d78d8 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:42.907897 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b419e33a-8d72-4b47-91ac-ab51c56d78d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:42.954219 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:43.004745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 737/1483] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:43.041497 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179e670-9c9a-4cdb-ad27-3c15f7d9a6d0 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:43.043057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e179e670-9c9a-4cdb-ad27-3c15f7d9a6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:43.070151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:43.164578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179e670-9c9a-4cdb-ad27-3c15f7d9a6d0 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:43.248911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c240dc-f640-4f33-8944-671236136aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:43.249592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c240dc-f640-4f33-8944-671236136aef service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:43.249865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c240dc-f640-4f33-8944-671236136aef service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:43.250141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c240dc-f640-4f33-8944-671236136aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:43.251802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-48c240dc-f640-4f33-8944-671236136aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:43.328991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:43.369030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c240dc-f640-4f33-8944-671236136aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:43.370519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c240dc-f640-4f33-8944-671236136aef service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:43.391127 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 747/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:43:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:43:45.483309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161215cd-4a52-4c52-a2cf-df23344ad200 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:45.483882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161215cd-4a52-4c52-a2cf-df23344ad200 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:45.483882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161215cd-4a52-4c52-a2cf-df23344ad200 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:45.485885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 738/1485] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:43:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:45.493975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71245b3d-3113-4251-8521-e10f3b8de60c None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:45.494469 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71245b3d-3113-4251-8521-e10f3b8de60c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:45.494469 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71245b3d-3113-4251-8521-e10f3b8de60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:45.574375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-71245b3d-3113-4251-8521-e10f3b8de60c None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:45.576212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-71245b3d-3113-4251-8521-e10f3b8de60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:45.681747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 748/1486] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:43:45.733198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161215cd-4a52-4c52-a2cf-df23344ad200 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:45.735313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-161215cd-4a52-4c52-a2cf-df23344ad200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:45.850480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161215cd-4a52-4c52-a2cf-df23344ad200 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:45.924200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:45.926158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd756e-0f6f-4242-bb5f-4ed9ce448094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:45.926949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd756e-0f6f-4242-bb5f-4ed9ce448094 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:45.927282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd756e-0f6f-4242-bb5f-4ed9ce448094 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:45.927593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd756e-0f6f-4242-bb5f-4ed9ce448094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:45.929861 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-02fd756e-0f6f-4242-bb5f-4ed9ce448094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:46.057975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd756e-0f6f-4242-bb5f-4ed9ce448094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:46.059431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd756e-0f6f-4242-bb5f-4ed9ce448094 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:46.087036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 739/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:43:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:43:48.295326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f54c0-bac7-4b65-a39d-2357f656b96b None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:48.295856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f54c0-bac7-4b65-a39d-2357f656b96b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:48.295856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f54c0-bac7-4b65-a39d-2357f656b96b None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:48.297323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 749/1488] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:48.303490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc65177-9532-4038-b9df-5d396a0522ea None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:48.303993 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc65177-9532-4038-b9df-5d396a0522ea None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:48.304142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc65177-9532-4038-b9df-5d396a0522ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:48.371888 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8bc65177-9532-4038-b9df-5d396a0522ea None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:48.373148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8bc65177-9532-4038-b9df-5d396a0522ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:48.432157 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:48.489325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 740/1489] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:43:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:48.533920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f54c0-bac7-4b65-a39d-2357f656b96b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:48.536478 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-782f54c0-bac7-4b65-a39d-2357f656b96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:48.583747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:48.660717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f54c0-bac7-4b65-a39d-2357f656b96b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:48.736343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89cab8-eb82-4f90-8735-5f7aa86e8453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:48.737011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89cab8-eb82-4f90-8735-5f7aa86e8453 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:48.737293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89cab8-eb82-4f90-8735-5f7aa86e8453 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:48.737568 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89cab8-eb82-4f90-8735-5f7aa86e8453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:48.739220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7e89cab8-eb82-4f90-8735-5f7aa86e8453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:48.833753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:48.866342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89cab8-eb82-4f90-8735-5f7aa86e8453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:48.867826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89cab8-eb82-4f90-8735-5f7aa86e8453 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:48.887020 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 750/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:43:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:43:51.177782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724303f-11cc-41d5-95c9-8f65d4245468 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:51.178393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724303f-11cc-41d5-95c9-8f65d4245468 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:51.178393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724303f-11cc-41d5-95c9-8f65d4245468 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:51.179741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 741/1491] 10.222.0.20 () {54 vars in 926 bytes} [Tue Jul 12 03:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:43:51.186342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304131c1-8607-400c-a564-5dd4f69bb9dc None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:51.187065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304131c1-8607-400c-a564-5dd4f69bb9dc None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:51.187235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304131c1-8607-400c-a564-5dd4f69bb9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:51.259847 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-304131c1-8607-400c-a564-5dd4f69bb9dc None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:43:51.261004 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-304131c1-8607-400c-a564-5dd4f69bb9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:51.367100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 751/1492] 10.222.0.20 () {58 vars in 1023 bytes} [Tue Jul 12 03:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:43:51.409537 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724303f-11cc-41d5-95c9-8f65d4245468 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:43:51.411811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4724303f-11cc-41d5-95c9-8f65d4245468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:51.532903 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724303f-11cc-41d5-95c9-8f65d4245468 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:43:51.617884 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4868cc-6a1f-4300-bf02-85b7905d8c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:43:51.618534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4868cc-6a1f-4300-bf02-85b7905d8c14 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:43:51.618860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4868cc-6a1f-4300-bf02-85b7905d8c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:43:51.619131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4868cc-6a1f-4300-bf02-85b7905d8c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:43:51.620797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9b4868cc-6a1f-4300-bf02-85b7905d8c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:43:51.630238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:43:51.735864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4868cc-6a1f-4300-bf02-85b7905d8c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:43:51.737354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4868cc-6a1f-4300-bf02-85b7905d8c14 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:43:51.758789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 742/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:43:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:44:01.364999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=82 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:44:01.754440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=50 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:45:27.166562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d78c45-6dbb-42d3-a141-c43449094950 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:27.167190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d78c45-6dbb-42d3-a141-c43449094950 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:27.167190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d78c45-6dbb-42d3-a141-c43449094950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:27.265145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-48d78c45-6dbb-42d3-a141-c43449094950 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:45:27.266851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-48d78c45-6dbb-42d3-a141-c43449094950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:27.375475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 752/1494] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:45:27.383935 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab478858-53e0-49c5-89bb-452b8d796aea None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:27.384481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab478858-53e0-49c5-89bb-452b8d796aea None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:27.384632 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab478858-53e0-49c5-89bb-452b8d796aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:27.467286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ab478858-53e0-49c5-89bb-452b8d796aea None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:45:27.468525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ab478858-53e0-49c5-89bb-452b8d796aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:27.567163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 743/1495] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:45:27.571992 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d78c45-6dbb-42d3-a141-c43449094950 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:45:27.573290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-48d78c45-6dbb-42d3-a141-c43449094950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:27.585112 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d78c45-6dbb-42d3-a141-c43449094950 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:45:27.649465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:45:27.686665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030a99b-3a65-4d49-a72b-fa4f904cb4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:45:27.687326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030a99b-3a65-4d49-a72b-fa4f904cb4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:27.687615 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030a99b-3a65-4d49-a72b-fa4f904cb4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:27.687882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030a99b-3a65-4d49-a72b-fa4f904cb4e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:27.688643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030a99b-3a65-4d49-a72b-fa4f904cb4e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:45:27.688949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030a99b-3a65-4d49-a72b-fa4f904cb4e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:45:27.690003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: WARNING py.warnings [None req-5030a99b-3a65-4d49-a72b-fa4f904cb4e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:45:27.690003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: warnings.warn(msg) Jul 12 03:45:27.690003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: Jul 12 03:45:27.690902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030a99b-3a65-4d49-a72b-fa4f904cb4e5 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:45:27.703084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 753/1496] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:45:27.711600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab478858-53e0-49c5-89bb-452b8d796aea None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:45:27.721244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab478858-53e0-49c5-89bb-452b8d796aea None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:45:27.789677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:45:27.825388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32395893-4e68-4ea7-8b12-c93593d3fe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:45:27.825983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32395893-4e68-4ea7-8b12-c93593d3fe51 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:27.826265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32395893-4e68-4ea7-8b12-c93593d3fe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:27.826537 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32395893-4e68-4ea7-8b12-c93593d3fe51 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:27.827226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32395893-4e68-4ea7-8b12-c93593d3fe51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:45:27.829142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-32395893-4e68-4ea7-8b12-c93593d3fe51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:45:27.829142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 12 03:45:27.829142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: Jul 12 03:45:27.830366 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32395893-4e68-4ea7-8b12-c93593d3fe51 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:45:27.896385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 744/1497] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:45:27 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:45:27.900998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030a99b-3a65-4d49-a72b-fa4f904cb4e5 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:45:27.902581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5030a99b-3a65-4d49-a72b-fa4f904cb4e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:27.937956 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030a99b-3a65-4d49-a72b-fa4f904cb4e5 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:45:28.036515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec2890-60ba-466e-bccf-a1d5e093970c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:45:28.037181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2890-60ba-466e-bccf-a1d5e093970c admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:28.037459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2890-60ba-466e-bccf-a1d5e093970c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:28.037729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2890-60ba-466e-bccf-a1d5e093970c admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:28.038425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec2890-60ba-466e-bccf-a1d5e093970c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:45:28.039443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: WARNING py.warnings [None req-deec2890-60ba-466e-bccf-a1d5e093970c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:45:28.039443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: warnings.warn(msg) Jul 12 03:45:28.039443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: Jul 12 03:45:28.040700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec2890-60ba-466e-bccf-a1d5e093970c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:45:28.083549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 754/1498] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:45:27 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:45:28.087681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32395893-4e68-4ea7-8b12-c93593d3fe51 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:45:28.089322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-32395893-4e68-4ea7-8b12-c93593d3fe51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:28.156659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32395893-4e68-4ea7-8b12-c93593d3fe51 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:45:28.250589 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:45:28.261984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8394b9-ead7-4205-b0cc-ed4d6dee547c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:45:28.262664 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8394b9-ead7-4205-b0cc-ed4d6dee547c admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:28.262944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8394b9-ead7-4205-b0cc-ed4d6dee547c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:28.263215 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8394b9-ead7-4205-b0cc-ed4d6dee547c admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:28.263951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8394b9-ead7-4205-b0cc-ed4d6dee547c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:45:28.265017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-9f8394b9-ead7-4205-b0cc-ed4d6dee547c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:45:28.265017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 12 03:45:28.265017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: Jul 12 03:45:28.265909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8394b9-ead7-4205-b0cc-ed4d6dee547c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:45:28.279154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 745/1499] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:45:28.283509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec2890-60ba-466e-bccf-a1d5e093970c admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:45:28.285068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-deec2890-60ba-466e-bccf-a1d5e093970c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:28.296809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec2890-60ba-466e-bccf-a1d5e093970c admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:45:28.323005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:45:28.397115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b379c-7884-4e47-a5e9-efd66f8c4200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:45:28.397697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b379c-7884-4e47-a5e9-efd66f8c4200 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:28.397962 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b379c-7884-4e47-a5e9-efd66f8c4200 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:28.398253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b379c-7884-4e47-a5e9-efd66f8c4200 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:28.398979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b379c-7884-4e47-a5e9-efd66f8c4200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:45:28.400042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: WARNING py.warnings [None req-2c1b379c-7884-4e47-a5e9-efd66f8c4200 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:45:28.400042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: warnings.warn(msg) Jul 12 03:45:28.400042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: Jul 12 03:45:28.400925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b379c-7884-4e47-a5e9-efd66f8c4200 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:45:28.413600 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 755/1500] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:45:28.418645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8394b9-ead7-4205-b0cc-ed4d6dee547c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:45:28.420621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9f8394b9-ead7-4205-b0cc-ed4d6dee547c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:28.456644 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8394b9-ead7-4205-b0cc-ed4d6dee547c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:45:28.553816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c4f39-d235-4f6c-837f-a651139ac145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:45:28.554743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c4f39-d235-4f6c-837f-a651139ac145 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:28.555022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c4f39-d235-4f6c-837f-a651139ac145 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:28.555297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c4f39-d235-4f6c-837f-a651139ac145 admin admin] PATH_INFO: `/v3/projects/ebd520db20be447dbd705967790c6857/users/730c5ba9fc3840a1adea3a068476fd4e/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:28.578328 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c4f39-d235-4f6c-837f-a651139ac145 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd520db20be447dbd705967790c6857, user_id=730c5ba9fc3840a1adea3a068476fd4e, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:45:28.579436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-e81c4f39-d235-4f6c-837f-a651139ac145 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:45:28.579436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 12 03:45:28.579436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: Jul 12 03:45:28.581493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c4f39-d235-4f6c-837f-a651139ac145 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:45:28.608850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 746/1501] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:45:28 2022] PUT /identity/v3/projects/ebd520db20be447dbd705967790c6857/users/730c5ba9fc3840a1adea3a068476fd4e/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:45:28.621317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dfed0-df45-42ac-b5ba-e1f7cb504bad None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:28.621609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dfed0-df45-42ac-b5ba-e1f7cb504bad None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:28.621886 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dfed0-df45-42ac-b5ba-e1f7cb504bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:28.683193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-305dfed0-df45-42ac-b5ba-e1f7cb504bad None None] MFA Rules not processed for user `730c5ba9fc3840a1adea3a068476fd4e`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:45:28.684666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-305dfed0-df45-42ac-b5ba-e1f7cb504bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:28.764624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 756/1502] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:45:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:45:28.778683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225aef-db50-4e3e-82ca-e0e45b52cccb None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:28.778976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225aef-db50-4e3e-82ca-e0e45b52cccb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:28.779247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225aef-db50-4e3e-82ca-e0e45b52cccb None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:28.780697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 747/1503] 10.222.0.20 () {54 vars in 963 bytes} [Tue Jul 12 03:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:45:28.787013 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83888a7-0e98-42a3-8bce-57eab1fb6265 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:28.787306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83888a7-0e98-42a3-8bce-57eab1fb6265 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:28.787592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83888a7-0e98-42a3-8bce-57eab1fb6265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:28.818761 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:45:28.884728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-b83888a7-0e98-42a3-8bce-57eab1fb6265 None None] MFA Rules not processed for user `cee6da907b614dc5b6ffb47e2860f459`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:45:28.886567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b83888a7-0e98-42a3-8bce-57eab1fb6265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:28.977265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 757/1504] 10.222.0.20 () {58 vars in 1060 bytes} [Tue Jul 12 03:45:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:45:28.985246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225aef-db50-4e3e-82ca-e0e45b52cccb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:45:28.986770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bf225aef-db50-4e3e-82ca-e0e45b52cccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:29.042075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:45:29.116566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225aef-db50-4e3e-82ca-e0e45b52cccb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:45:29.193229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6faf92-4d3a-447a-9ae6-eded6bfdcf6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:45:29.193909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6faf92-4d3a-447a-9ae6-eded6bfdcf6d service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:29.194222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6faf92-4d3a-447a-9ae6-eded6bfdcf6d service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:29.194496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6faf92-4d3a-447a-9ae6-eded6bfdcf6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:29.196177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fb6faf92-4d3a-447a-9ae6-eded6bfdcf6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:29.307425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6faf92-4d3a-447a-9ae6-eded6bfdcf6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:45:29.308908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6faf92-4d3a-447a-9ae6-eded6bfdcf6d service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:45:29.312328 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:45:29.328196 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 748/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:45:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:45:29.373002 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019103bc-b2d7-4004-abbd-9f16664b51b5 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:29.373473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019103bc-b2d7-4004-abbd-9f16664b51b5 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:29.373573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019103bc-b2d7-4004-abbd-9f16664b51b5 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:29.374856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 758/1506] 10.222.0.20 () {52 vars in 829 bytes} [Tue Jul 12 03:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:45:29.378838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6faf92-4d3a-447a-9ae6-eded6bfdcf6d service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:45:29.380109 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fb6faf92-4d3a-447a-9ae6-eded6bfdcf6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:29.495477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6faf92-4d3a-447a-9ae6-eded6bfdcf6d service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:45:29.596564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:45:29.597361 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:29.597636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:29.597957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] PATH_INFO: `/v3/users/730c5ba9fc3840a1adea3a068476fd4e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:29.610758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=730c5ba9fc3840a1adea3a068476fd4e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:45:29.611791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:45:29.611791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 12 03:45:29.611791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: Jul 12 03:45:29.613051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:45:29.623743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:45:29.682984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '730c5ba9fc3840a1adea3a068476fd4e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:45:29.693220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '730c5ba9fc3840a1adea3a068476fd4e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:45:29.693673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '730c5ba9fc3840a1adea3a068476fd4e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:45:29.706535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '730c5ba9fc3840a1adea3a068476fd4e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:45:29.724991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '730c5ba9fc3840a1adea3a068476fd4e'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:45:29.731002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 749/1507] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:45:29 2022] DELETE /identity/v3/users/730c5ba9fc3840a1adea3a068476fd4e => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:45:29.735677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79444518-1a20-4782-b35a-2266b2c6c954 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:29.735966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79444518-1a20-4782-b35a-2266b2c6c954 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:29.736305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79444518-1a20-4782-b35a-2266b2c6c954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:29.797105 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-79444518-1a20-4782-b35a-2266b2c6c954 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:45:29.798375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-79444518-1a20-4782-b35a-2266b2c6c954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:29.877426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:45:29.891630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 759/1508] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:45:29.902016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:45:29.903289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:30.010170 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9f5c1-d7e4-4042-896d-52b7eb140d91 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:45:30.196023 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:45:30.218886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c067ca-055e-48e4-a236-605ba07aa9a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:45:30.219552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c067ca-055e-48e4-a236-605ba07aa9a3 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:30.219831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c067ca-055e-48e4-a236-605ba07aa9a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:30.220140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c067ca-055e-48e4-a236-605ba07aa9a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:30.221771 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-88c067ca-055e-48e4-a236-605ba07aa9a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:30.338030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c067ca-055e-48e4-a236-605ba07aa9a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:45:30.339486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c067ca-055e-48e4-a236-605ba07aa9a3 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:45:30.358465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 750/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:45:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:45:30.738779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79444518-1a20-4782-b35a-2266b2c6c954 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:45:30.739991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-79444518-1a20-4782-b35a-2266b2c6c954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:45:30.752327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79444518-1a20-4782-b35a-2266b2c6c954 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:45:30.855349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:45:30.856262 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:45:30.856550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:45:30.856827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] PATH_INFO: `/v3/projects/ebd520db20be447dbd705967790c6857` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:45:30.867565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd520db20be447dbd705967790c6857)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:45:30.868656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: WARNING py.warnings [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:45:30.868656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: warnings.warn(msg) Jul 12 03:45:30.868656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: Jul 12 03:45:30.869883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:45:30.941546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd520db20be447dbd705967790c6857', 'request_id': 'req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:45:30.953638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd520db20be447dbd705967790c6857', 'request_id': 'req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:45:30.967327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd520db20be447dbd705967790c6857', 'request_id': 'req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:45:30.990082 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] The token cache is being invalidate because project ebd520db20be447dbd705967790c6857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:45:30.990272 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:45:30.990731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c17a9504-f121-4bd7-ab9a-3afd7f944e7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:45:30.992271 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 760/1510] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:45:30 2022] DELETE /identity/v3/projects/ebd520db20be447dbd705967790c6857 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:45:40.355988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=62 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:45:40.987965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:47:06.530507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f869f6e-f46d-473b-b217-967b502ea643 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:06.531114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f869f6e-f46d-473b-b217-967b502ea643 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:06.531114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f869f6e-f46d-473b-b217-967b502ea643 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:06.532534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 03:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:47:06.538760 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359afb85-183e-47d5-b20f-8c689142a5fa None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:06.539518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359afb85-183e-47d5-b20f-8c689142a5fa None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:06.539673 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359afb85-183e-47d5-b20f-8c689142a5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:06.631559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-359afb85-183e-47d5-b20f-8c689142a5fa None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:47:06.632823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-359afb85-183e-47d5-b20f-8c689142a5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:06.747657 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 03:47:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:47:06.901573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f869f6e-f46d-473b-b217-967b502ea643 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:47:06.905046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0f869f6e-f46d-473b-b217-967b502ea643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:07.038406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f869f6e-f46d-473b-b217-967b502ea643 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:47:07.044135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab9632-fdac-4aae-a867-c109862f9b88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:47:07.044787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab9632-fdac-4aae-a867-c109862f9b88 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:07.045069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab9632-fdac-4aae-a867-c109862f9b88 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:07.045343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab9632-fdac-4aae-a867-c109862f9b88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:07.053873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab9632-fdac-4aae-a867-c109862f9b88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:47:07.055481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab9632-fdac-4aae-a867-c109862f9b88 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:47:07.065398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 752/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:47:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:47:07.888161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd80f0-a982-49b2-b019-33df18ce9001 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:07.889142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd80f0-a982-49b2-b019-33df18ce9001 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:07.889142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd80f0-a982-49b2-b019-33df18ce9001 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:07.891009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 03:47:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:47:07.896371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e01e0-7c09-41c6-b397-8f08ac966ed3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:07.896863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e01e0-7c09-41c6-b397-8f08ac966ed3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:07.896863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e01e0-7c09-41c6-b397-8f08ac966ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:07.917452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b66e01e0-7c09-41c6-b397-8f08ac966ed3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:47:07.918668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b66e01e0-7c09-41c6-b397-8f08ac966ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:07.927781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 03:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:47:08.086638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd80f0-a982-49b2-b019-33df18ce9001 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:47:08.108968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd80f0-a982-49b2-b019-33df18ce9001 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:47:08.115191 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf4b34-ec2f-4c42-bd42-b7e1e31d0d3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:47:08.115860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf4b34-ec2f-4c42-bd42-b7e1e31d0d3d service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:08.116317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf4b34-ec2f-4c42-bd42-b7e1e31d0d3d service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:08.116620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf4b34-ec2f-4c42-bd42-b7e1e31d0d3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:08.125033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf4b34-ec2f-4c42-bd42-b7e1e31d0d3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:47:08.126702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf4b34-ec2f-4c42-bd42-b7e1e31d0d3d service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:47:08.136375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 763/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:47:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:47:08.462460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f47633-78a2-4ee5-aead-9573f4f0e040 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:08.462722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f47633-78a2-4ee5-aead-9573f4f0e040 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:08.463026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f47633-78a2-4ee5-aead-9573f4f0e040 None None] PATH_INFO: `/v3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:08.464394 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 754/1517] 10.222.0.44 () {54 vars in 925 bytes} [Tue Jul 12 03:47:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:47:08.471887 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af6dce-e95f-4800-96ec-582d3aa76039 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:08.472212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af6dce-e95f-4800-96ec-582d3aa76039 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:08.472486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af6dce-e95f-4800-96ec-582d3aa76039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:08.534906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-09af6dce-e95f-4800-96ec-582d3aa76039 None None] MFA Rules not processed for user `eded06390a2744c9aa52f77e3bfc2db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:47:08.536136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-09af6dce-e95f-4800-96ec-582d3aa76039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:08.553947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:47:08.612782 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 764/1518] 10.222.0.44 () {58 vars in 1015 bytes} [Tue Jul 12 03:47:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:47:17.684100 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887a9d9-7f0d-489d-a15b-fa1d8cc480a5 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:17.684577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887a9d9-7f0d-489d-a15b-fa1d8cc480a5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:17.684659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887a9d9-7f0d-489d-a15b-fa1d8cc480a5 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:17.686033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 755/1519] 10.222.0.20 () {54 vars in 960 bytes} [Tue Jul 12 03:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:47:17.691080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae608325-b34b-4b31-b043-2d0e295579f3 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:17.691470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae608325-b34b-4b31-b043-2d0e295579f3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:17.691626 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae608325-b34b-4b31-b043-2d0e295579f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:17.711094 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-ae608325-b34b-4b31-b043-2d0e295579f3 None None] MFA Rules not processed for user `eded06390a2744c9aa52f77e3bfc2db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:47:17.713010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ae608325-b34b-4b31-b043-2d0e295579f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:17.723053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 765/1520] 10.222.0.20 () {58 vars in 1057 bytes} [Tue Jul 12 03:47:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:47:17.730885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887a9d9-7f0d-489d-a15b-fa1d8cc480a5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:47:17.747080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887a9d9-7f0d-489d-a15b-fa1d8cc480a5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:47:17.752909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b6243-4e10-4ffc-afb5-5b3cfc25c85c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:47:17.753562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b6243-4e10-4ffc-afb5-5b3cfc25c85c service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:17.753848 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b6243-4e10-4ffc-afb5-5b3cfc25c85c service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:17.754121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b6243-4e10-4ffc-afb5-5b3cfc25c85c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:17.762990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b6243-4e10-4ffc-afb5-5b3cfc25c85c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:47:17.764654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b6243-4e10-4ffc-afb5-5b3cfc25c85c service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:47:17.775008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 756/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:47:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:47:19.374175 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091df83-8c95-470b-804d-5ed2eca9afdd None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:19.374818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091df83-8c95-470b-804d-5ed2eca9afdd None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:19.374818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091df83-8c95-470b-804d-5ed2eca9afdd None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:19.376141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 03:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:47:19.382011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28713865-0961-4b43-a660-9fec4cb84ff2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:19.382697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28713865-0961-4b43-a660-9fec4cb84ff2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:19.382896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28713865-0961-4b43-a660-9fec4cb84ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:19.406715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-28713865-0961-4b43-a660-9fec4cb84ff2 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:47:19.407922 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-28713865-0961-4b43-a660-9fec4cb84ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:19.417064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 03:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:47:19.571238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091df83-8c95-470b-804d-5ed2eca9afdd None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:47:19.587909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091df83-8c95-470b-804d-5ed2eca9afdd None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:47:19.594161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689803b4-817c-482d-9e8c-7988a37a8335 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:47:19.594796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689803b4-817c-482d-9e8c-7988a37a8335 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:19.595076 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689803b4-817c-482d-9e8c-7988a37a8335 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:19.595870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689803b4-817c-482d-9e8c-7988a37a8335 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:19.604322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689803b4-817c-482d-9e8c-7988a37a8335 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:47:19.605926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689803b4-817c-482d-9e8c-7988a37a8335 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:47:19.615385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 767/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:47:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:47:20.437349 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda97eb7-ef41-42ef-ba90-7414f1f80c76 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:20.437859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda97eb7-ef41-42ef-ba90-7414f1f80c76 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:20.437859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda97eb7-ef41-42ef-ba90-7414f1f80c76 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:20.439285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 03:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:47:20.444542 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba291e6-0273-4970-9939-6f84872ee4e3 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:20.445107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba291e6-0273-4970-9939-6f84872ee4e3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:20.445107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba291e6-0273-4970-9939-6f84872ee4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:20.466056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2ba291e6-0273-4970-9939-6f84872ee4e3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:47:20.467238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2ba291e6-0273-4970-9939-6f84872ee4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:20.476889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 03:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:47:20.643387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda97eb7-ef41-42ef-ba90-7414f1f80c76 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:47:20.665509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda97eb7-ef41-42ef-ba90-7414f1f80c76 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:47:20.671722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e33434-e98d-4ce0-9e8a-67e88253e27c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:47:20.672398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e33434-e98d-4ce0-9e8a-67e88253e27c service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:20.672677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e33434-e98d-4ce0-9e8a-67e88253e27c service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:20.672950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e33434-e98d-4ce0-9e8a-67e88253e27c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:20.682016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e33434-e98d-4ce0-9e8a-67e88253e27c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:47:20.683621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e33434-e98d-4ce0-9e8a-67e88253e27c service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:47:20.693364 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 759/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:47:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:47:24.474649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6226de-8dca-4e06-bd3f-0c8a30c3212c None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:24.475566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6226de-8dca-4e06-bd3f-0c8a30c3212c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:24.475566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6226de-8dca-4e06-bd3f-0c8a30c3212c None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:24.477773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:47:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:47:24.484159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1376007-0b5a-4b05-affb-0d782b4d20f6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:24.484902 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1376007-0b5a-4b05-affb-0d782b4d20f6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:24.485040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1376007-0b5a-4b05-affb-0d782b4d20f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:24.505384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a1376007-0b5a-4b05-affb-0d782b4d20f6 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:47:24.506587 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a1376007-0b5a-4b05-affb-0d782b4d20f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:24.515655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:47:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:47:24.597967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6226de-8dca-4e06-bd3f-0c8a30c3212c None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:47:24.601162 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7c6226de-8dca-4e06-bd3f-0c8a30c3212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:24.691528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6226de-8dca-4e06-bd3f-0c8a30c3212c None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:47:24.695658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2f1de-ae9e-437c-81e1-f2631c43fda2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:47:24.696346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2f1de-ae9e-437c-81e1-f2631c43fda2 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:24.696632 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2f1de-ae9e-437c-81e1-f2631c43fda2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:24.696908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2f1de-ae9e-437c-81e1-f2631c43fda2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:24.707133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2f1de-ae9e-437c-81e1-f2631c43fda2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:47:24.708616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2f1de-ae9e-437c-81e1-f2631c43fda2 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:47:24.719348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 770/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:47:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:47:26.718701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d5fe2-7d07-49a0-830c-69c7a06a1247 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:26.719574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d5fe2-7d07-49a0-830c-69c7a06a1247 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:26.719574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d5fe2-7d07-49a0-830c-69c7a06a1247 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:26.721650 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:47:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:47:26.729563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303cd11-1717-4f49-aedd-96449dd36186 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:26.730007 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303cd11-1717-4f49-aedd-96449dd36186 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:26.730118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303cd11-1717-4f49-aedd-96449dd36186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:26.753471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-c303cd11-1717-4f49-aedd-96449dd36186 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:47:26.755250 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c303cd11-1717-4f49-aedd-96449dd36186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:26.765429 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:47:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:47:26.805551 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d5fe2-7d07-49a0-830c-69c7a06a1247 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:47:26.808930 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-df2d5fe2-7d07-49a0-830c-69c7a06a1247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:26.832258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d5fe2-7d07-49a0-830c-69c7a06a1247 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:47:26.836877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdafbe90-5c44-47c7-937b-d431566ad9c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:47:26.837523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafbe90-5c44-47c7-937b-d431566ad9c3 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:26.837813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafbe90-5c44-47c7-937b-d431566ad9c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:26.838083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdafbe90-5c44-47c7-937b-d431566ad9c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:26.848419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafbe90-5c44-47c7-937b-d431566ad9c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:47:26.849840 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdafbe90-5c44-47c7-937b-d431566ad9c3 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:47:26.859745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 762/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:47:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:47:30.296988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee127c-d254-461a-825f-d51b0efa58db None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:30.297556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee127c-d254-461a-825f-d51b0efa58db None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:30.297556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee127c-d254-461a-825f-d51b0efa58db None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:30.298852 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:47:30.304327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5b9e3-3510-4c39-9a1d-2938b28ce7e7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:30.304809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5b9e3-3510-4c39-9a1d-2938b28ce7e7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:30.304920 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5b9e3-3510-4c39-9a1d-2938b28ce7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:30.324360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-84f5b9e3-3510-4c39-9a1d-2938b28ce7e7 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:47:30.325554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-84f5b9e3-3510-4c39-9a1d-2938b28ce7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:30.333994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:47:30.633012 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee127c-d254-461a-825f-d51b0efa58db None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:47:30.635330 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d5ee127c-d254-461a-825f-d51b0efa58db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:30.707385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:47:30.721901 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee127c-d254-461a-825f-d51b0efa58db None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:47:30.726360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e74149-739e-4a3e-88d9-efbd5d5d6a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:47:30.727011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e74149-739e-4a3e-88d9-efbd5d5d6a1b service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:30.727295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e74149-739e-4a3e-88d9-efbd5d5d6a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:30.727597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e74149-739e-4a3e-88d9-efbd5d5d6a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:30.737608 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e74149-739e-4a3e-88d9-efbd5d5d6a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:47:30.739058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e74149-739e-4a3e-88d9-efbd5d5d6a1b service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:47:30.748935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 773/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:47:32.793457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb383bd-01cd-4513-9e8a-8bcaab96d72c None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:32.794003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb383bd-01cd-4513-9e8a-8bcaab96d72c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:32.794003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb383bd-01cd-4513-9e8a-8bcaab96d72c None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:32.795310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 03:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:47:32.801616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b36b30-41e8-4585-81c2-4276d0c170e9 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:32.802064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b36b30-41e8-4585-81c2-4276d0c170e9 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:32.802155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b36b30-41e8-4585-81c2-4276d0c170e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:32.823961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-91b36b30-41e8-4585-81c2-4276d0c170e9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:47:32.825753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-91b36b30-41e8-4585-81c2-4276d0c170e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:32.835766 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 03:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:47:33.150739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb383bd-01cd-4513-9e8a-8bcaab96d72c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:47:33.153200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-feb383bd-01cd-4513-9e8a-8bcaab96d72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:47:33.161941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=84 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:47:33.169561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb383bd-01cd-4513-9e8a-8bcaab96d72c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:47:33.176680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea073ee9-71e9-4aa6-89db-1d4de3b02d5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:47:33.177665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea073ee9-71e9-4aa6-89db-1d4de3b02d5b service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:47:33.178128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea073ee9-71e9-4aa6-89db-1d4de3b02d5b service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:47:33.178521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea073ee9-71e9-4aa6-89db-1d4de3b02d5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:47:33.194048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea073ee9-71e9-4aa6-89db-1d4de3b02d5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:47:33.195481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea073ee9-71e9-4aa6-89db-1d4de3b02d5b service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:47:33.205289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 765/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:47:42.817560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=11 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:47:43.201360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:01.659108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6845fe6-6210-42ff-b7ca-1b6eb35fe76d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:01.660590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6845fe6-6210-42ff-b7ca-1b6eb35fe76d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:01.661121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6845fe6-6210-42ff-b7ca-1b6eb35fe76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:01.669791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e309bc-6853-45e5-9937-5460d9256397 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:01.670402 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e309bc-6853-45e5-9937-5460d9256397 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:01.670402 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e309bc-6853-45e5-9937-5460d9256397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:01.686015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f6845fe6-6210-42ff-b7ca-1b6eb35fe76d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:01.687643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f6845fe6-6210-42ff-b7ca-1b6eb35fe76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:01.702147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 775/1540] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:01.703364 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-34e309bc-6853-45e5-9937-5460d9256397 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:01.704614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-34e309bc-6853-45e5-9937-5460d9256397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:01.712398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595103d9-3859-429a-bf94-e10b57e5621d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:01.712690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595103d9-3859-429a-bf94-e10b57e5621d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:01.712954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595103d9-3859-429a-bf94-e10b57e5621d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:01.719359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 766/1541] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:01.731929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120f912-28e1-4273-97a6-b547e5c28387 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:01.732309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120f912-28e1-4273-97a6-b547e5c28387 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:01.732517 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120f912-28e1-4273-97a6-b547e5c28387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:01.745020 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-595103d9-3859-429a-bf94-e10b57e5621d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:01.746324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-595103d9-3859-429a-bf94-e10b57e5621d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:01.762450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 776/1542] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:01.763964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a09c5-4ed0-4f06-a611-df64f6f57618 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:01.764277 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a09c5-4ed0-4f06-a611-df64f6f57618 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:01.764553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a09c5-4ed0-4f06-a611-df64f6f57618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:01.768362 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8120f912-28e1-4273-97a6-b547e5c28387 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:01.769578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8120f912-28e1-4273-97a6-b547e5c28387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:01.785511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 767/1543] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:01.786251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120f912-28e1-4273-97a6-b547e5c28387 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:01.800154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120f912-28e1-4273-97a6-b547e5c28387 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:01.801142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d95a09c5-4ed0-4f06-a611-df64f6f57618 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:01.802348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d95a09c5-4ed0-4f06-a611-df64f6f57618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:01.815979 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d220293-e980-427d-9e5b-0ef9db13a8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:01.816775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 777/1544] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:01.817293 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d220293-e980-427d-9e5b-0ef9db13a8ca admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:01.817578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d220293-e980-427d-9e5b-0ef9db13a8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:01.817849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d220293-e980-427d-9e5b-0ef9db13a8ca admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:01.818061 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5737e-1414-4026-ad83-70d0c7569b75 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:01.818353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5737e-1414-4026-ad83-70d0c7569b75 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:01.818627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5737e-1414-4026-ad83-70d0c7569b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:01.818859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d220293-e980-427d-9e5b-0ef9db13a8ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:01.819171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d220293-e980-427d-9e5b-0ef9db13a8ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:01.820256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-7d220293-e980-427d-9e5b-0ef9db13a8ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:49:01.820256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 12 03:49:01.820256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: Jul 12 03:49:01.823191 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d220293-e980-427d-9e5b-0ef9db13a8ca admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:01.841873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 768/1545] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:01.843742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d220293-e980-427d-9e5b-0ef9db13a8ca admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:01.854436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-aad5737e-1414-4026-ad83-70d0c7569b75 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:01.855008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d220293-e980-427d-9e5b-0ef9db13a8ca admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:01.855637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-aad5737e-1414-4026-ad83-70d0c7569b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:01.874399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 778/1546] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:01.875522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42474b21-ba74-49a1-89f5-41b754df8dbb None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:01.875808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42474b21-ba74-49a1-89f5-41b754df8dbb None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:01.876199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42474b21-ba74-49a1-89f5-41b754df8dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:01.879561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddee9c-73bd-4fc3-9391-ec5a13df4aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:01.880281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddee9c-73bd-4fc3-9391-ec5a13df4aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:01.880580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddee9c-73bd-4fc3-9391-ec5a13df4aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:01.880839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddee9c-73bd-4fc3-9391-ec5a13df4aa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:01.881571 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddee9c-73bd-4fc3-9391-ec5a13df4aa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:01.881885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddee9c-73bd-4fc3-9391-ec5a13df4aa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:01.883602 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddee9c-73bd-4fc3-9391-ec5a13df4aa3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:01.906731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 769/1547] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:01.915768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddee9c-73bd-4fc3-9391-ec5a13df4aa3 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:01.926746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-42474b21-ba74-49a1-89f5-41b754df8dbb None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:01.928855 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-42474b21-ba74-49a1-89f5-41b754df8dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:01.933776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddee9c-73bd-4fc3-9391-ec5a13df4aa3 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:01.949624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 779/1548] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:01.950569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42474b21-ba74-49a1-89f5-41b754df8dbb None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:01.951116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f67bc-58bf-4336-85d9-efc077d82025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:01.951808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f67bc-58bf-4336-85d9-efc077d82025 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:01.952122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f67bc-58bf-4336-85d9-efc077d82025 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:01.952392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f67bc-58bf-4336-85d9-efc077d82025 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:01.953091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f67bc-58bf-4336-85d9-efc077d82025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:01.955015 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f67bc-58bf-4336-85d9-efc077d82025 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:01.962112 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42474b21-ba74-49a1-89f5-41b754df8dbb None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:01.967640 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cfb1a-40f9-4728-8b8a-68b7587cd9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:01.968436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cfb1a-40f9-4728-8b8a-68b7587cd9dd admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:01.968795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cfb1a-40f9-4728-8b8a-68b7587cd9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:01.969142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cfb1a-40f9-4728-8b8a-68b7587cd9dd admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:01.970027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cfb1a-40f9-4728-8b8a-68b7587cd9dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:01.970319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cfb1a-40f9-4728-8b8a-68b7587cd9dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:01.972032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cfb1a-40f9-4728-8b8a-68b7587cd9dd admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:01.985053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 780/1549] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:01.985983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cfb1a-40f9-4728-8b8a-68b7587cd9dd admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:01.997590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cfb1a-40f9-4728-8b8a-68b7587cd9dd admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.003135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5096a-112a-469e-b78b-fc4bd2e55488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.003988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5096a-112a-469e-b78b-fc4bd2e55488 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.004457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5096a-112a-469e-b78b-fc4bd2e55488 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.004623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 770/1550] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:01 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:02.005092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5096a-112a-469e-b78b-fc4bd2e55488 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.005839 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5096a-112a-469e-b78b-fc4bd2e55488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.007032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: WARNING py.warnings [None req-bad5096a-112a-469e-b78b-fc4bd2e55488 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:49:02.007032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: warnings.warn(msg) Jul 12 03:49:02.007032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: Jul 12 03:49:02.008424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5096a-112a-469e-b78b-fc4bd2e55488 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.009979 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a7f47-d8e4-4a95-aeb6-1d155fd377a4 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.010368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a7f47-d8e4-4a95-aeb6-1d155fd377a4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.010646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a7f47-d8e4-4a95-aeb6-1d155fd377a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.042871 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1e0a7f47-d8e4-4a95-aeb6-1d155fd377a4 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:02.044132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1e0a7f47-d8e4-4a95-aeb6-1d155fd377a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:02.047380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 781/1551] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:01 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:02.048931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42602c98-5095-451e-957b-2b107d64af1a None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.049229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42602c98-5095-451e-957b-2b107d64af1a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.049502 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42602c98-5095-451e-957b-2b107d64af1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.068378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-42602c98-5095-451e-957b-2b107d64af1a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:02.069617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-42602c98-5095-451e-957b-2b107d64af1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:02.105553 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 771/1552] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:02.106844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd9b39-c72f-4a27-ab7d-8ab9b5f8080f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.107131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd9b39-c72f-4a27-ab7d-8ab9b5f8080f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.107403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd9b39-c72f-4a27-ab7d-8ab9b5f8080f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.125045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-64fd9b39-c72f-4a27-ab7d-8ab9b5f8080f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:02.126340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-64fd9b39-c72f-4a27-ab7d-8ab9b5f8080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:02.128237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 782/1553] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:02.129056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42602c98-5095-451e-957b-2b107d64af1a None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.138291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42602c98-5095-451e-957b-2b107d64af1a None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.139167 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 772/1554] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:02.140137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd9b39-c72f-4a27-ab7d-8ab9b5f8080f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.143235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220f565-6b35-4d5c-a71c-288e47969379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.143891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220f565-6b35-4d5c-a71c-288e47969379 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.144206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220f565-6b35-4d5c-a71c-288e47969379 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.144480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220f565-6b35-4d5c-a71c-288e47969379 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.145171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220f565-6b35-4d5c-a71c-288e47969379 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.147181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220f565-6b35-4d5c-a71c-288e47969379 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.149987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd9b39-c72f-4a27-ab7d-8ab9b5f8080f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.154930 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366d269-0383-4f51-9cad-42c2e7e042ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.155586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366d269-0383-4f51-9cad-42c2e7e042ed admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.155872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366d269-0383-4f51-9cad-42c2e7e042ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.156180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366d269-0383-4f51-9cad-42c2e7e042ed admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.156864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366d269-0383-4f51-9cad-42c2e7e042ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.157964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-8366d269-0383-4f51-9cad-42c2e7e042ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:49:02.157964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 12 03:49:02.157964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: Jul 12 03:49:02.159157 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366d269-0383-4f51-9cad-42c2e7e042ed admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.174979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 783/1555] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:02.176216 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901b961-38a6-4977-a1ee-0dbf2232be0c None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.176497 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901b961-38a6-4977-a1ee-0dbf2232be0c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.176889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901b961-38a6-4977-a1ee-0dbf2232be0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.179169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 773/1556] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/users => generated 292 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:02.180179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366d269-0383-4f51-9cad-42c2e7e042ed admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.189293 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366d269-0383-4f51-9cad-42c2e7e042ed admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.195207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2901b961-38a6-4977-a1ee-0dbf2232be0c None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:02.196476 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2901b961-38a6-4977-a1ee-0dbf2232be0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:02.244956 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca7b7e-754e-4d5a-ada1-02a8051b8b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.245617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca7b7e-754e-4d5a-ada1-02a8051b8b2b admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.245897 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca7b7e-754e-4d5a-ada1-02a8051b8b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.246165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca7b7e-754e-4d5a-ada1-02a8051b8b2b admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.246843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca7b7e-754e-4d5a-ada1-02a8051b8b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.248813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca7b7e-754e-4d5a-ada1-02a8051b8b2b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.257636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 784/1557] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:02.259101 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda4c5c-f6e7-462c-ae70-1369873535d3 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.259496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda4c5c-f6e7-462c-ae70-1369873535d3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.259875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda4c5c-f6e7-462c-ae70-1369873535d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.265469 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:02.277786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 774/1558] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:02.281005 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750088be-967c-442b-9398-899264b17507 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.281005 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750088be-967c-442b-9398-899264b17507 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.281005 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750088be-967c-442b-9398-899264b17507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.282699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-cfda4c5c-f6e7-462c-ae70-1369873535d3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:02.283908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-cfda4c5c-f6e7-462c-ae70-1369873535d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:02.291970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 785/1559] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:02.291970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51f634-2f4f-4741-bc0b-687485156d7a None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.291970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51f634-2f4f-4741-bc0b-687485156d7a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.291970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51f634-2f4f-4741-bc0b-687485156d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.300735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:02.302578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-750088be-967c-442b-9398-899264b17507 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:02.303990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-750088be-967c-442b-9398-899264b17507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:02.312290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8f51f634-2f4f-4741-bc0b-687485156d7a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:02.316241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 775/1560] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:02.316369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8f51f634-2f4f-4741-bc0b-687485156d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:02.316453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750088be-967c-442b-9398-899264b17507 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.321294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 786/1561] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:02.322049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51f634-2f4f-4741-bc0b-687485156d7a None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.323051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750088be-967c-442b-9398-899264b17507 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.327751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376869b0-adff-4090-9466-f6e7c6ddaf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.328438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376869b0-adff-4090-9466-f6e7c6ddaf95 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.328725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376869b0-adff-4090-9466-f6e7c6ddaf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.329001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376869b0-adff-4090-9466-f6e7c6ddaf95 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.329730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376869b0-adff-4090-9466-f6e7c6ddaf95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:02.330036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376869b0-adff-4090-9466-f6e7c6ddaf95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.331233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51f634-2f4f-4741-bc0b-687485156d7a None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.331654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376869b0-adff-4090-9466-f6e7c6ddaf95 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.336018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708706e-4470-4988-9531-3802fc43c74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.336690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708706e-4470-4988-9531-3802fc43c74d admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.336973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708706e-4470-4988-9531-3802fc43c74d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.337241 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708706e-4470-4988-9531-3802fc43c74d admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.337914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708706e-4470-4988-9531-3802fc43c74d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.339817 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708706e-4470-4988-9531-3802fc43c74d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.343326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 776/1562] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:02.344165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376869b0-adff-4090-9466-f6e7c6ddaf95 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.356235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376869b0-adff-4090-9466-f6e7c6ddaf95 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.361156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f9b20-bcd8-4dc9-b64a-5766b41e5bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.361813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f9b20-bcd8-4dc9-b64a-5766b41e5bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.362101 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f9b20-bcd8-4dc9-b64a-5766b41e5bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.362229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 787/1563] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/users => generated 297 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:02.362394 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f9b20-bcd8-4dc9-b64a-5766b41e5bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.363243 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f9b20-bcd8-4dc9-b64a-5766b41e5bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.363502 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5d8d4-baf7-42b2-adab-dafe99117a82 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.363788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5d8d4-baf7-42b2-adab-dafe99117a82 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.364085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5d8d4-baf7-42b2-adab-dafe99117a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.364891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f9b20-bcd8-4dc9-b64a-5766b41e5bc1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.375796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 777/1564] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:02.376442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f9b20-bcd8-4dc9-b64a-5766b41e5bc1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.384925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f9b5d8d4-baf7-42b2-adab-dafe99117a82 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:02.386969 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f9b5d8d4-baf7-42b2-adab-dafe99117a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:02.387861 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f9b20-bcd8-4dc9-b64a-5766b41e5bc1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.393212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c35315-ad32-4312-81ae-6d1fea909c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.393921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c35315-ad32-4312-81ae-6d1fea909c7e admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.394207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c35315-ad32-4312-81ae-6d1fea909c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.394507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c35315-ad32-4312-81ae-6d1fea909c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.395237 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c35315-ad32-4312-81ae-6d1fea909c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.396883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c35315-ad32-4312-81ae-6d1fea909c7e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.397177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 788/1565] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:02.401405 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385621f0-c2bb-4a4e-9ae6-bf5944bc766b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.401405 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385621f0-c2bb-4a4e-9ae6-bf5944bc766b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.401405 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385621f0-c2bb-4a4e-9ae6-bf5944bc766b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.411490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 778/1566] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:02.412810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c35315-ad32-4312-81ae-6d1fea909c7e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.417683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-385621f0-c2bb-4a4e-9ae6-bf5944bc766b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:02.418883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-385621f0-c2bb-4a4e-9ae6-bf5944bc766b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:02.422290 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c35315-ad32-4312-81ae-6d1fea909c7e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.426833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63319db-c511-404b-ba87-387550f55668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.427407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63319db-c511-404b-ba87-387550f55668 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.427689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63319db-c511-404b-ba87-387550f55668 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.427964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63319db-c511-404b-ba87-387550f55668 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.428201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 789/1567] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:02.428701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63319db-c511-404b-ba87-387550f55668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:02.429036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385621f0-c2bb-4a4e-9ae6-bf5944bc766b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.429148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63319db-c511-404b-ba87-387550f55668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.430725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63319db-c511-404b-ba87-387550f55668 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.438889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385621f0-c2bb-4a4e-9ae6-bf5944bc766b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.442759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 779/1568] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:02.443478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63319db-c511-404b-ba87-387550f55668 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.444199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d2452-58cb-42d4-9658-2d8fbf29b18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.445050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d2452-58cb-42d4-9658-2d8fbf29b18a admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.445430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d2452-58cb-42d4-9658-2d8fbf29b18a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.445707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d2452-58cb-42d4-9658-2d8fbf29b18a admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.446431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d2452-58cb-42d4-9658-2d8fbf29b18a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:02.446740 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d2452-58cb-42d4-9658-2d8fbf29b18a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.448397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d2452-58cb-42d4-9658-2d8fbf29b18a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.453170 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63319db-c511-404b-ba87-387550f55668 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.458052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2f36f-f066-4c9b-8ea6-dc7d5ad8d406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.458668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2f36f-f066-4c9b-8ea6-dc7d5ad8d406 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.458969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2f36f-f066-4c9b-8ea6-dc7d5ad8d406 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.459237 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2f36f-f066-4c9b-8ea6-dc7d5ad8d406 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.459932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2f36f-f066-4c9b-8ea6-dc7d5ad8d406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.461034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 790/1569] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:02.461924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2f36f-f066-4c9b-8ea6-dc7d5ad8d406 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.468821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d2452-58cb-42d4-9658-2d8fbf29b18a admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.478777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d2452-58cb-42d4-9658-2d8fbf29b18a admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.484206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f4387-d8d2-4772-92fd-d519648a2e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.484890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f4387-d8d2-4772-92fd-d519648a2e4c admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.485169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f4387-d8d2-4772-92fd-d519648a2e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.485448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f4387-d8d2-4772-92fd-d519648a2e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.486166 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f4387-d8d2-4772-92fd-d519648a2e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.487775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f4387-d8d2-4772-92fd-d519648a2e4c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.489995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 780/1570] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/projects => generated 355 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:02.490803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2f36f-f066-4c9b-8ea6-dc7d5ad8d406 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.498785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 791/1571] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:02.499743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f4387-d8d2-4772-92fd-d519648a2e4c admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.509068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f4387-d8d2-4772-92fd-d519648a2e4c admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.558658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2f36f-f066-4c9b-8ea6-dc7d5ad8d406 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.563629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023567c-a5a3-4e53-9e5d-5f42ce62fe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.564353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023567c-a5a3-4e53-9e5d-5f42ce62fe54 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.564642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023567c-a5a3-4e53-9e5d-5f42ce62fe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.564913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023567c-a5a3-4e53-9e5d-5f42ce62fe54 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.565672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023567c-a5a3-4e53-9e5d-5f42ce62fe54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.567335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023567c-a5a3-4e53-9e5d-5f42ce62fe54 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.570817 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626591c6-d849-4b3c-8bbd-cf4eb89225b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.571493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626591c6-d849-4b3c-8bbd-cf4eb89225b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.571780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626591c6-d849-4b3c-8bbd-cf4eb89225b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.572082 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626591c6-d849-4b3c-8bbd-cf4eb89225b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.572814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626591c6-d849-4b3c-8bbd-cf4eb89225b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:02.573124 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626591c6-d849-4b3c-8bbd-cf4eb89225b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.574742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626591c6-d849-4b3c-8bbd-cf4eb89225b7 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.577450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 781/1572] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:02.578277 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023567c-a5a3-4e53-9e5d-5f42ce62fe54 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.587249 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 792/1573] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:02.587871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626591c6-d849-4b3c-8bbd-cf4eb89225b7 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.588169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023567c-a5a3-4e53-9e5d-5f42ce62fe54 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.595809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b03f16-d79d-44ea-8941-1267ad81240b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.596529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b03f16-d79d-44ea-8941-1267ad81240b admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.596815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b03f16-d79d-44ea-8941-1267ad81240b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.597114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b03f16-d79d-44ea-8941-1267ad81240b admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.597838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b03f16-d79d-44ea-8941-1267ad81240b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.599197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626591c6-d849-4b3c-8bbd-cf4eb89225b7 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.599447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b03f16-d79d-44ea-8941-1267ad81240b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.603953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e308c-844e-462e-9c2b-032ddcda158e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.604820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e308c-844e-462e-9c2b-032ddcda158e admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.605110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e308c-844e-462e-9c2b-032ddcda158e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.605394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e308c-844e-462e-9c2b-032ddcda158e admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.606115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e308c-844e-462e-9c2b-032ddcda158e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:02.606427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e308c-844e-462e-9c2b-032ddcda158e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.608114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e308c-844e-462e-9c2b-032ddcda158e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.610428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 782/1574] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:02.611206 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b03f16-d79d-44ea-8941-1267ad81240b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.620590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b03f16-d79d-44ea-8941-1267ad81240b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.623619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 793/1575] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:02.624506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e308c-844e-462e-9c2b-032ddcda158e admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.625408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9a707-fb78-4c4f-9577-a120128d6922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.626107 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9a707-fb78-4c4f-9577-a120128d6922 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.626445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9a707-fb78-4c4f-9577-a120128d6922 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.626776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9a707-fb78-4c4f-9577-a120128d6922 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.627521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9a707-fb78-4c4f-9577-a120128d6922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.629512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9a707-fb78-4c4f-9577-a120128d6922 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.634056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e308c-844e-462e-9c2b-032ddcda158e admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.638639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0369204-c930-4d1e-9dc5-d4378d68eec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.639320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0369204-c930-4d1e-9dc5-d4378d68eec1 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.639612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0369204-c930-4d1e-9dc5-d4378d68eec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.639930 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0369204-c930-4d1e-9dc5-d4378d68eec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.640659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0369204-c930-4d1e-9dc5-d4378d68eec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.642575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0369204-c930-4d1e-9dc5-d4378d68eec1 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.660785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 783/1576] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:02.660785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9a707-fb78-4c4f-9577-a120128d6922 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.675496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 794/1577] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:02.675496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0369204-c930-4d1e-9dc5-d4378d68eec1 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.704894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:02.725525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9a707-fb78-4c4f-9577-a120128d6922 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.737024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0369204-c930-4d1e-9dc5-d4378d68eec1 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.741694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b79f79-c7e6-46ec-8358-43ad6da50745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.742415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b79f79-c7e6-46ec-8358-43ad6da50745 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.742699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b79f79-c7e6-46ec-8358-43ad6da50745 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.742976 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b79f79-c7e6-46ec-8358-43ad6da50745 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.743708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b79f79-c7e6-46ec-8358-43ad6da50745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.745373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b79f79-c7e6-46ec-8358-43ad6da50745 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.747452 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:02.756914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 795/1578] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:02.757872 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b79f79-c7e6-46ec-8358-43ad6da50745 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.767989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b79f79-c7e6-46ec-8358-43ad6da50745 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.772560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf7a99-7053-44dc-9a0b-7dfca37f768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.773884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf7a99-7053-44dc-9a0b-7dfca37f768f admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.774159 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf7a99-7053-44dc-9a0b-7dfca37f768f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.774440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf7a99-7053-44dc-9a0b-7dfca37f768f admin admin] PATH_INFO: `/v3/projects/3440ac7f7cb74b0292e846cdcd91c1d1/users/249f1cf3635a45ec8181a2a7e2ef1c1c/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.785620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd860f9c-6f05-424b-8413-5f520025b7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.786298 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd860f9c-6f05-424b-8413-5f520025b7d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.786587 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd860f9c-6f05-424b-8413-5f520025b7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.786867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd860f9c-6f05-424b-8413-5f520025b7d4 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.787619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd860f9c-6f05-424b-8413-5f520025b7d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.788847 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf7a99-7053-44dc-9a0b-7dfca37f768f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3440ac7f7cb74b0292e846cdcd91c1d1, user_id=249f1cf3635a45ec8181a2a7e2ef1c1c, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.789551 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd860f9c-6f05-424b-8413-5f520025b7d4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.789983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: WARNING py.warnings [None req-f0cf7a99-7053-44dc-9a0b-7dfca37f768f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:49:02.789983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: warnings.warn(msg) Jul 12 03:49:02.789983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: Jul 12 03:49:02.792107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf7a99-7053-44dc-9a0b-7dfca37f768f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.806226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 796/1579] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:02 2022] PUT /identity/v3/projects/3440ac7f7cb74b0292e846cdcd91c1d1/users/249f1cf3635a45ec8181a2a7e2ef1c1c/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:02.807012 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf7a99-7053-44dc-9a0b-7dfca37f768f admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.814326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 784/1580] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/users => generated 278 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:02.814326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd860f9c-6f05-424b-8413-5f520025b7d4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.822359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd860f9c-6f05-424b-8413-5f520025b7d4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.870755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf7a99-7053-44dc-9a0b-7dfca37f768f admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.875401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f2cf6-504a-4f28-88cc-fc748f483fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.876095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f2cf6-504a-4f28-88cc-fc748f483fed admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.876387 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f2cf6-504a-4f28-88cc-fc748f483fed admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.876667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f2cf6-504a-4f28-88cc-fc748f483fed admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.877374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f2cf6-504a-4f28-88cc-fc748f483fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.879135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11ec66-e580-47bc-9868-857cc75e35a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.879319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f2cf6-504a-4f28-88cc-fc748f483fed admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.880050 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11ec66-e580-47bc-9868-857cc75e35a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.880345 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11ec66-e580-47bc-9868-857cc75e35a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.880625 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11ec66-e580-47bc-9868-857cc75e35a3 admin admin] PATH_INFO: `/v3/projects/e9417b8011914e108ae00b5aa583ce50/users/bf67549bf7e443c6b77eaf71fb6b709c/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.895651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11ec66-e580-47bc-9868-857cc75e35a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9417b8011914e108ae00b5aa583ce50, user_id=bf67549bf7e443c6b77eaf71fb6b709c, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.899085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11ec66-e580-47bc-9868-857cc75e35a3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.909377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 797/1581] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:02.909377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f2cf6-504a-4f28-88cc-fc748f483fed admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.912518 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 785/1582] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:02 2022] PUT /identity/v3/projects/e9417b8011914e108ae00b5aa583ce50/users/bf67549bf7e443c6b77eaf71fb6b709c/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:02.913290 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11ec66-e580-47bc-9868-857cc75e35a3 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:02.974769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11ec66-e580-47bc-9868-857cc75e35a3 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.975761 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f2cf6-504a-4f28-88cc-fc748f483fed admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:02.979400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061cf36-cc91-4ee0-ac55-0eb93d8bb367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.980108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061cf36-cc91-4ee0-ac55-0eb93d8bb367 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.980394 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061cf36-cc91-4ee0-ac55-0eb93d8bb367 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.980496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ada48c-0ec5-4ebc-857a-562ada974990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:02.980670 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061cf36-cc91-4ee0-ac55-0eb93d8bb367 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.981135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ada48c-0ec5-4ebc-857a-562ada974990 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:02.981356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061cf36-cc91-4ee0-ac55-0eb93d8bb367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.981469 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ada48c-0ec5-4ebc-857a-562ada974990 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:02.981698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ada48c-0ec5-4ebc-857a-562ada974990 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:02.982411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ada48c-0ec5-4ebc-857a-562ada974990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:02.983285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061cf36-cc91-4ee0-ac55-0eb93d8bb367 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.984357 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ada48c-0ec5-4ebc-857a-562ada974990 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:02.991212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:03.005524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 786/1583] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:03.006332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061cf36-cc91-4ee0-ac55-0eb93d8bb367 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.016465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 798/1584] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:02 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:03.016465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ada48c-0ec5-4ebc-857a-562ada974990 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.019095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061cf36-cc91-4ee0-ac55-0eb93d8bb367 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.061340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:03.075447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7967903-1093-4c75-b52f-0ad185fcddaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.076292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7967903-1093-4c75-b52f-0ad185fcddaa admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.076578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7967903-1093-4c75-b52f-0ad185fcddaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.076854 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7967903-1093-4c75-b52f-0ad185fcddaa admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.077547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7967903-1093-4c75-b52f-0ad185fcddaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.078934 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ada48c-0ec5-4ebc-857a-562ada974990 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.079515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7967903-1093-4c75-b52f-0ad185fcddaa admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.083692 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c60a9-0fc0-446b-a2c4-1254b4e8843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.084704 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c60a9-0fc0-446b-a2c4-1254b4e8843c admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.084994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c60a9-0fc0-446b-a2c4-1254b4e8843c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.085273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c60a9-0fc0-446b-a2c4-1254b4e8843c admin admin] PATH_INFO: `/v3/projects/fd03cff94a1842d8811deafa0dce10f9/users/4ac2825db0c649cab6dcc2e27fbaf80d/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.100280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 787/1585] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:03 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:03.101360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7967903-1093-4c75-b52f-0ad185fcddaa admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.101751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c60a9-0fc0-446b-a2c4-1254b4e8843c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd03cff94a1842d8811deafa0dce10f9, user_id=4ac2825db0c649cab6dcc2e27fbaf80d, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.104553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c60a9-0fc0-446b-a2c4-1254b4e8843c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.111350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7967903-1093-4c75-b52f-0ad185fcddaa admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.118557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 799/1586] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:03 2022] PUT /identity/v3/projects/fd03cff94a1842d8811deafa0dce10f9/users/4ac2825db0c649cab6dcc2e27fbaf80d/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:03.122285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734a512-6098-4625-9676-fbcdbfe9798b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.122285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734a512-6098-4625-9676-fbcdbfe9798b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.122285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734a512-6098-4625-9676-fbcdbfe9798b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.146030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2734a512-6098-4625-9676-fbcdbfe9798b None None] MFA Rules not processed for user `249f1cf3635a45ec8181a2a7e2ef1c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:03.147243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2734a512-6098-4625-9676-fbcdbfe9798b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:03.170202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ba975-fc38-47a0-b6a6-53ef1ce42734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.170909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ba975-fc38-47a0-b6a6-53ef1ce42734 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.171196 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ba975-fc38-47a0-b6a6-53ef1ce42734 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.171468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ba975-fc38-47a0-b6a6-53ef1ce42734 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.172222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ba975-fc38-47a0-b6a6-53ef1ce42734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.173924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ba975-fc38-47a0-b6a6-53ef1ce42734 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.183760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 788/1587] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:03.184645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ba975-fc38-47a0-b6a6-53ef1ce42734 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.193984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ba975-fc38-47a0-b6a6-53ef1ce42734 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.198733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bb738-a20b-48af-b2cd-9c4a7a711393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.199369 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bb738-a20b-48af-b2cd-9c4a7a711393 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.199653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bb738-a20b-48af-b2cd-9c4a7a711393 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.199927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bb738-a20b-48af-b2cd-9c4a7a711393 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.200634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bb738-a20b-48af-b2cd-9c4a7a711393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.202534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bb738-a20b-48af-b2cd-9c4a7a711393 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.214231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 800/1588] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:03.215547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d962e-c06e-4fd6-bc66-58db61d5cafa None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.215759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d962e-c06e-4fd6-bc66-58db61d5cafa None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.216071 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d962e-c06e-4fd6-bc66-58db61d5cafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.230220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 789/1589] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:03 2022] POST /identity/v3/users => generated 282 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:03.231055 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bb738-a20b-48af-b2cd-9c4a7a711393 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.233379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-ce9d962e-c06e-4fd6-bc66-58db61d5cafa None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:03.234523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ce9d962e-c06e-4fd6-bc66-58db61d5cafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:03.241410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bb738-a20b-48af-b2cd-9c4a7a711393 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.244970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 801/1590] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:03.245880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d962e-c06e-4fd6-bc66-58db61d5cafa None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.246756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa494bb6-e92d-43cb-8ebd-356648676898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.247435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa494bb6-e92d-43cb-8ebd-356648676898 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.247721 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa494bb6-e92d-43cb-8ebd-356648676898 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.248025 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa494bb6-e92d-43cb-8ebd-356648676898 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.249342 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa494bb6-e92d-43cb-8ebd-356648676898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.250694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa494bb6-e92d-43cb-8ebd-356648676898 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.258182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d962e-c06e-4fd6-bc66-58db61d5cafa None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.263617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f54b18-2977-44dd-8b78-e2f7e5331ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.264364 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 790/1591] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:03.264508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f54b18-2977-44dd-8b78-e2f7e5331ced admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.264922 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f54b18-2977-44dd-8b78-e2f7e5331ced admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.265211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f54b18-2977-44dd-8b78-e2f7e5331ced admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.265908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f54b18-2977-44dd-8b78-e2f7e5331ced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.268073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f54b18-2977-44dd-8b78-e2f7e5331ced admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.268294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa494bb6-e92d-43cb-8ebd-356648676898 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.278402 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa494bb6-e92d-43cb-8ebd-356648676898 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.283016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c507ba-420c-4a6c-b53d-7e14382a5d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.283681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c507ba-420c-4a6c-b53d-7e14382a5d50 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.284055 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c507ba-420c-4a6c-b53d-7e14382a5d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.284302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c507ba-420c-4a6c-b53d-7e14382a5d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.284977 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c507ba-420c-4a6c-b53d-7e14382a5d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.286570 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c507ba-420c-4a6c-b53d-7e14382a5d50 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.292391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 802/1592] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:03 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:03.292391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5795da-16bb-4cb3-87a0-7aa7bc5b9c6b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.292391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5795da-16bb-4cb3-87a0-7aa7bc5b9c6b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.292391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5795da-16bb-4cb3-87a0-7aa7bc5b9c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.297891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 791/1593] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:03.298774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c507ba-420c-4a6c-b53d-7e14382a5d50 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.308895 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8e5795da-16bb-4cb3-87a0-7aa7bc5b9c6b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:03.309327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c507ba-420c-4a6c-b53d-7e14382a5d50 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.310090 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8e5795da-16bb-4cb3-87a0-7aa7bc5b9c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:03.314096 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca13d43-35c0-4542-ae7d-4ccbc64188dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.314851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca13d43-35c0-4542-ae7d-4ccbc64188dc admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.315236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca13d43-35c0-4542-ae7d-4ccbc64188dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.315610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca13d43-35c0-4542-ae7d-4ccbc64188dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.316583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca13d43-35c0-4542-ae7d-4ccbc64188dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.318107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 803/1594] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:03.318431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca13d43-35c0-4542-ae7d-4ccbc64188dc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.318879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5795da-16bb-4cb3-87a0-7aa7bc5b9c6b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.329035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5795da-16bb-4cb3-87a0-7aa7bc5b9c6b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.334033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 792/1595] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:03.334033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa97e0c-38d5-4469-b5f2-7491efed7294 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.334033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa97e0c-38d5-4469-b5f2-7491efed7294 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.334033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa97e0c-38d5-4469-b5f2-7491efed7294 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.334452 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d446bc7-9564-4f4a-b5de-32cbdd2a5e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.334651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 793/1596] 10.222.0.20 () {54 vars in 961 bytes} [Tue Jul 12 03:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:49:03.334977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d446bc7-9564-4f4a-b5de-32cbdd2a5e80 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.335262 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d446bc7-9564-4f4a-b5de-32cbdd2a5e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.335369 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa97e0c-38d5-4469-b5f2-7491efed7294 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.335756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d446bc7-9564-4f4a-b5de-32cbdd2a5e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.336546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d446bc7-9564-4f4a-b5de-32cbdd2a5e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.338147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d446bc7-9564-4f4a-b5de-32cbdd2a5e80 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.349774 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 804/1597] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:03.350583 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d446bc7-9564-4f4a-b5de-32cbdd2a5e80 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.360078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d446bc7-9564-4f4a-b5de-32cbdd2a5e80 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.364932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e5bcc-16c2-442a-8e6e-529057b13c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.365578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e5bcc-16c2-442a-8e6e-529057b13c74 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.365866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e5bcc-16c2-442a-8e6e-529057b13c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.366143 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e5bcc-16c2-442a-8e6e-529057b13c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.366539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:03.367103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e5bcc-16c2-442a-8e6e-529057b13c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.368949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e5bcc-16c2-442a-8e6e-529057b13c74 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.379107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 805/1598] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:03.379885 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e5bcc-16c2-442a-8e6e-529057b13c74 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.389575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e5bcc-16c2-442a-8e6e-529057b13c74 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.393619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa97e0c-38d5-4469-b5f2-7491efed7294 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.394089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea186064-5068-4375-ae3c-7662a4599bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.394729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea186064-5068-4375-ae3c-7662a4599bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.395014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea186064-5068-4375-ae3c-7662a4599bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.395295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea186064-5068-4375-ae3c-7662a4599bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.396017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea186064-5068-4375-ae3c-7662a4599bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.397136 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0775952-64a5-4fcc-bab5-f7913870f84a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.397623 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea186064-5068-4375-ae3c-7662a4599bc3 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.397784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0775952-64a5-4fcc-bab5-f7913870f84a service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.398530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0775952-64a5-4fcc-bab5-f7913870f84a service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.398812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0775952-64a5-4fcc-bab5-f7913870f84a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.407555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0775952-64a5-4fcc-bab5-f7913870f84a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.408895 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 806/1599] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:03.409799 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea186064-5068-4375-ae3c-7662a4599bc3 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.411880 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0775952-64a5-4fcc-bab5-f7913870f84a service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.419647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea186064-5068-4375-ae3c-7662a4599bc3 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.421445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 794/1600] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:03.422229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0775952-64a5-4fcc-bab5-f7913870f84a service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.424518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5e7e3-0b73-4a67-af5c-1f02a70c2fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.425149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5e7e3-0b73-4a67-af5c-1f02a70c2fea admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.425553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5e7e3-0b73-4a67-af5c-1f02a70c2fea admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.425701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5e7e3-0b73-4a67-af5c-1f02a70c2fea admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.426407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5e7e3-0b73-4a67-af5c-1f02a70c2fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.428025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5e7e3-0b73-4a67-af5c-1f02a70c2fea admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.430333 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0775952-64a5-4fcc-bab5-f7913870f84a service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.434519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e000c-c0d9-4f1e-9da6-296d20993f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.435159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e000c-c0d9-4f1e-9da6-296d20993f63 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.435458 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e000c-c0d9-4f1e-9da6-296d20993f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.435726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e000c-c0d9-4f1e-9da6-296d20993f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.439390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 807/1601] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:03.440398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5e7e3-0b73-4a67-af5c-1f02a70c2fea admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.448925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e000c-c0d9-4f1e-9da6-296d20993f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.450653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e000c-c0d9-4f1e-9da6-296d20993f63 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.451530 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5e7e3-0b73-4a67-af5c-1f02a70c2fea admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.456809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b3de3-1d1f-43d0-aff4-60de9d718cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.457790 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b3de3-1d1f-43d0-aff4-60de9d718cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.458076 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b3de3-1d1f-43d0-aff4-60de9d718cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.458370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b3de3-1d1f-43d0-aff4-60de9d718cf4 admin admin] PATH_INFO: `/v3/projects/be14eac72d2342bca5a603236e4441b2/users/4448db1fb05841e88e527e44c4a23b3f/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.462232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 795/1602] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:03.463739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd083f5c-795c-4871-8d23-a7182b3899d0 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.464290 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd083f5c-795c-4871-8d23-a7182b3899d0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.464730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd083f5c-795c-4871-8d23-a7182b3899d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.475033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b3de3-1d1f-43d0-aff4-60de9d718cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be14eac72d2342bca5a603236e4441b2, user_id=4448db1fb05841e88e527e44c4a23b3f, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.477865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b3de3-1d1f-43d0-aff4-60de9d718cf4 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.495888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 808/1603] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:03 2022] PUT /identity/v3/projects/be14eac72d2342bca5a603236e4441b2/users/4448db1fb05841e88e527e44c4a23b3f/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:03.496867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b3de3-1d1f-43d0-aff4-60de9d718cf4 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.518877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dd083f5c-795c-4871-8d23-a7182b3899d0 None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:03.520189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dd083f5c-795c-4871-8d23-a7182b3899d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:03.525273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:03.576367 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b3de3-1d1f-43d0-aff4-60de9d718cf4 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.581496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdf0d6-9c46-47cc-ab31-9fa71cd5582d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.583168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdf0d6-9c46-47cc-ab31-9fa71cd5582d admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.583969 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdf0d6-9c46-47cc-ab31-9fa71cd5582d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.584499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdf0d6-9c46-47cc-ab31-9fa71cd5582d admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.586303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdf0d6-9c46-47cc-ab31-9fa71cd5582d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.588348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdf0d6-9c46-47cc-ab31-9fa71cd5582d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.599954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 809/1604] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:03.600750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdf0d6-9c46-47cc-ab31-9fa71cd5582d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.605517 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 796/1605] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Jul 12 03:49:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:03.606535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd083f5c-795c-4871-8d23-a7182b3899d0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.615322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdf0d6-9c46-47cc-ab31-9fa71cd5582d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.624462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984391b-fe9e-4afd-bbe5-52ca9998d0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.625711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984391b-fe9e-4afd-bbe5-52ca9998d0f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.626210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984391b-fe9e-4afd-bbe5-52ca9998d0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.626683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984391b-fe9e-4afd-bbe5-52ca9998d0f3 admin admin] PATH_INFO: `/v3/projects/38b640d5f6734a6b95b46a6f9dd695ec/users/70ea64f67f264b20a7c556c625a64f80/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.633690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd083f5c-795c-4871-8d23-a7182b3899d0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.643383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc7b31-fab8-46da-9700-f0dcaf14dd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.644358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc7b31-fab8-46da-9700-f0dcaf14dd06 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.645029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc7b31-fab8-46da-9700-f0dcaf14dd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.645401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc7b31-fab8-46da-9700-f0dcaf14dd06 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.646443 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc7b31-fab8-46da-9700-f0dcaf14dd06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.649699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc7b31-fab8-46da-9700-f0dcaf14dd06 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.653065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984391b-fe9e-4afd-bbe5-52ca9998d0f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b640d5f6734a6b95b46a6f9dd695ec, user_id=70ea64f67f264b20a7c556c625a64f80, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.655939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984391b-fe9e-4afd-bbe5-52ca9998d0f3 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.662453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 797/1606] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:03.663285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc7b31-fab8-46da-9700-f0dcaf14dd06 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.674944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc7b31-fab8-46da-9700-f0dcaf14dd06 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.678486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 810/1607] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:03 2022] PUT /identity/v3/projects/38b640d5f6734a6b95b46a6f9dd695ec/users/70ea64f67f264b20a7c556c625a64f80/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:03.680894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603740b-6c74-42c3-8014-2e42b94a8237 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.680894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603740b-6c74-42c3-8014-2e42b94a8237 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.681157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603740b-6c74-42c3-8014-2e42b94a8237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.709427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8603740b-6c74-42c3-8014-2e42b94a8237 None None] MFA Rules not processed for user `4448db1fb05841e88e527e44c4a23b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:03.710637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8603740b-6c74-42c3-8014-2e42b94a8237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:03.759705 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3ce1c-d606-43e3-8521-64885d35499b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.760678 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3ce1c-d606-43e3-8521-64885d35499b admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.760956 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3ce1c-d606-43e3-8521-64885d35499b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.761227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3ce1c-d606-43e3-8521-64885d35499b admin admin] PATH_INFO: `/v3/projects/dcd9e1e7c72b4f6fa85068bce0cf35eb/users/17a48195a14a40c4b4e6420636252783/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.777983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3ce1c-d606-43e3-8521-64885d35499b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcd9e1e7c72b4f6fa85068bce0cf35eb, user_id=17a48195a14a40c4b4e6420636252783, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.780758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3ce1c-d606-43e3-8521-64885d35499b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.795756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 798/1608] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:03 2022] PUT /identity/v3/projects/dcd9e1e7c72b4f6fa85068bce0cf35eb/users/17a48195a14a40c4b4e6420636252783/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:03.796523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 811/1609] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:03 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:03.797384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603740b-6c74-42c3-8014-2e42b94a8237 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.806112 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3ce1c-d606-43e3-8521-64885d35499b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.817273 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:03.882815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603740b-6c74-42c3-8014-2e42b94a8237 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.888877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3ce1c-d606-43e3-8521-64885d35499b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:03.895184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c724fc-580a-4678-a4b2-167ad86a5c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.896050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:03.896568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c724fc-580a-4678-a4b2-167ad86a5c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.896854 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c724fc-580a-4678-a4b2-167ad86a5c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.897132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c724fc-580a-4678-a4b2-167ad86a5c0e admin admin] PATH_INFO: `/v3/projects/a9b3d9a1b53b429daedcb93600ef183e/users/2ca986b3bf7f4877badb5828db87dc4c/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:03.915242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c724fc-580a-4678-a4b2-167ad86a5c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b3d9a1b53b429daedcb93600ef183e, user_id=2ca986b3bf7f4877badb5828db87dc4c, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:03.917993 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c724fc-580a-4678-a4b2-167ad86a5c0e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:03.934027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 799/1610] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:03 2022] PUT /identity/v3/projects/a9b3d9a1b53b429daedcb93600ef183e/users/2ca986b3bf7f4877badb5828db87dc4c/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:03.936971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c724fc-580a-4678-a4b2-167ad86a5c0e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:03.975824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446cd1f7-df73-4bb6-9dd2-5a1777bea740 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:03.976767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cd1f7-df73-4bb6-9dd2-5a1777bea740 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:03.977147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cd1f7-df73-4bb6-9dd2-5a1777bea740 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:03.977487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cd1f7-df73-4bb6-9dd2-5a1777bea740 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.021800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c724fc-580a-4678-a4b2-167ad86a5c0e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:04.031829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc8b67-4e8a-4021-b0c5-9f67aa4fb40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:04.033456 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc8b67-4e8a-4021-b0c5-9f67aa4fb40b admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.034588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc8b67-4e8a-4021-b0c5-9f67aa4fb40b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.035942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc8b67-4e8a-4021-b0c5-9f67aa4fb40b admin admin] PATH_INFO: `/v3/projects/60bc2d22a5ee48d09fa2ef945593079c/users/ea57984dbdb442cb92b801d48586f176/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.047275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446cd1f7-df73-4bb6-9dd2-5a1777bea740 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:04.050174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446cd1f7-df73-4bb6-9dd2-5a1777bea740 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:04.059553 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc8b67-4e8a-4021-b0c5-9f67aa4fb40b admin admin] RBAC: Authorizing `identity:create_grant(project_id=60bc2d22a5ee48d09fa2ef945593079c, user_id=ea57984dbdb442cb92b801d48586f176, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:04.062554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc8b67-4e8a-4021-b0c5-9f67aa4fb40b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:04.073077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 812/1611] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:04.078368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028e340-f79d-4052-b549-be50621c42ab None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.078935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028e340-f79d-4052-b549-be50621c42ab None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.079477 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028e340-f79d-4052-b549-be50621c42ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.096660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 800/1612] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:03 2022] PUT /identity/v3/projects/60bc2d22a5ee48d09fa2ef945593079c/users/ea57984dbdb442cb92b801d48586f176/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:04.112214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc8b67-4e8a-4021-b0c5-9f67aa4fb40b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:04.116572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-0028e340-f79d-4052-b549-be50621c42ab None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:04.118981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0028e340-f79d-4052-b549-be50621c42ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:04.223268 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc8b67-4e8a-4021-b0c5-9f67aa4fb40b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:04.228671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1452c76-74df-44e3-8e32-e1d7580f0175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:04.229563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1452c76-74df-44e3-8e32-e1d7580f0175 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.230038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1452c76-74df-44e3-8e32-e1d7580f0175 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.230471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1452c76-74df-44e3-8e32-e1d7580f0175 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.231424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1452c76-74df-44e3-8e32-e1d7580f0175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:04.233835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1452c76-74df-44e3-8e32-e1d7580f0175 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:04.242679 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 813/1613] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:04.253693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ecc39-5c1f-436f-bd98-e586f6f768d3 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.254347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ecc39-5c1f-436f-bd98-e586f6f768d3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.254905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ecc39-5c1f-436f-bd98-e586f6f768d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.267222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:04.279714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 801/1614] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:04 2022] POST /identity/v3/projects => generated 355 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:04.281140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a494c5c-753a-4446-92ab-fd4c53a1913c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.281434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a494c5c-753a-4446-92ab-fd4c53a1913c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.282342 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a494c5c-753a-4446-92ab-fd4c53a1913c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.319503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-950ecc39-5c1f-436f-bd98-e586f6f768d3 None None] MFA Rules not processed for user `17a48195a14a40c4b4e6420636252783`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:04.320894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-950ecc39-5c1f-436f-bd98-e586f6f768d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:04.339186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3a494c5c-753a-4446-92ab-fd4c53a1913c None None] MFA Rules not processed for user `2ca986b3bf7f4877badb5828db87dc4c`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:04.341345 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3a494c5c-753a-4446-92ab-fd4c53a1913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:04.413099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:04.439440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 814/1615] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:04 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:04.440871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172ebf6-7817-423a-a0f6-e982d6c4a082 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.441161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172ebf6-7817-423a-a0f6-e982d6c4a082 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.441436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172ebf6-7817-423a-a0f6-e982d6c4a082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.472576 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8172ebf6-7817-423a-a0f6-e982d6c4a082 None None] MFA Rules not processed for user `ea57984dbdb442cb92b801d48586f176`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:04.474410 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8172ebf6-7817-423a-a0f6-e982d6c4a082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:04.518852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 802/1616] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:04 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:04.520431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a494c5c-753a-4446-92ab-fd4c53a1913c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:04.581927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 815/1617] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:04.581927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172ebf6-7817-423a-a0f6-e982d6c4a082 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:04.649307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a494c5c-753a-4446-92ab-fd4c53a1913c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:04.658178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e72741-5cc9-483a-a387-6523be3241e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:04.660197 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e72741-5cc9-483a-a387-6523be3241e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.660938 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e72741-5cc9-483a-a387-6523be3241e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.661446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e72741-5cc9-483a-a387-6523be3241e3 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.662311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e72741-5cc9-483a-a387-6523be3241e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:04.664946 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172ebf6-7817-423a-a0f6-e982d6c4a082 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:04.665183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e72741-5cc9-483a-a387-6523be3241e3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:04.670614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eaf0fc-e39b-414c-bcd5-339b0112ebb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:04.671405 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaf0fc-e39b-414c-bcd5-339b0112ebb9 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.671696 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaf0fc-e39b-414c-bcd5-339b0112ebb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.672089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eaf0fc-e39b-414c-bcd5-339b0112ebb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.686781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eaf0fc-e39b-414c-bcd5-339b0112ebb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:04.691165 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eaf0fc-e39b-414c-bcd5-339b0112ebb9 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:04.697088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 803/1618] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:04 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:04.698990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e72741-5cc9-483a-a387-6523be3241e3 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:04.706034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 816/1619] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:04.707463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a228af8-027c-4b70-89cb-4be3807aa282 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.707797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a228af8-027c-4b70-89cb-4be3807aa282 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.708484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a228af8-027c-4b70-89cb-4be3807aa282 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.714609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e72741-5cc9-483a-a387-6523be3241e3 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:04.715222 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 817/1620] 10.222.0.20 () {54 vars in 912 bytes} [Tue Jul 12 03:49:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:49:04.718389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a228af8-027c-4b70-89cb-4be3807aa282 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:04.725946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93528015-8967-424f-aa5a-96f3436daa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:04.727016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93528015-8967-424f-aa5a-96f3436daa83 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.727538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93528015-8967-424f-aa5a-96f3436daa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.727970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93528015-8967-424f-aa5a-96f3436daa83 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.729812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93528015-8967-424f-aa5a-96f3436daa83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:04.732378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93528015-8967-424f-aa5a-96f3436daa83 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:04.734192 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a228af8-027c-4b70-89cb-4be3807aa282 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:04.743786 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb76a98-a7a8-4a31-ae9b-ad79a1e6ec29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:04.747295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb76a98-a7a8-4a31-ae9b-ad79a1e6ec29 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.747860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb76a98-a7a8-4a31-ae9b-ad79a1e6ec29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.748570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb76a98-a7a8-4a31-ae9b-ad79a1e6ec29 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.749886 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb76a98-a7a8-4a31-ae9b-ad79a1e6ec29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:04.760306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb76a98-a7a8-4a31-ae9b-ad79a1e6ec29 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:04.786921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 804/1621] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:04 2022] POST /identity/v3/projects => generated 405 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:04.788522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2d815-3e37-4707-aaab-aaa52eb5e6aa None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.788941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2d815-3e37-4707-aaab-aaa52eb5e6aa None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.789353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f2d815-3e37-4707-aaab-aaa52eb5e6aa None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.794805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 818/1622] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:04 2022] POST /identity/v3/projects => generated 363 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:04.794947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 805/1623] 10.222.0.20 () {54 vars in 961 bytes} [Tue Jul 12 03:49:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:49:04.795057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb76a98-a7a8-4a31-ae9b-ad79a1e6ec29 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:04.795137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dca6c-8512-401a-83e1-8987b408891c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.795137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dca6c-8512-401a-83e1-8987b408891c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.795137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043dca6c-8512-401a-83e1-8987b408891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.825481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-043dca6c-8512-401a-83e1-8987b408891c None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:04.826727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-043dca6c-8512-401a-83e1-8987b408891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:04.909760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:04.918666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:04.924483 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb76a98-a7a8-4a31-ae9b-ad79a1e6ec29 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:04.936790 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb56f66-53ec-4639-b409-e131068fe430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:04.938092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb56f66-53ec-4639-b409-e131068fe430 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.941493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb56f66-53ec-4639-b409-e131068fe430 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.941759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb56f66-53ec-4639-b409-e131068fe430 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.942701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb56f66-53ec-4639-b409-e131068fe430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:04.944513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb56f66-53ec-4639-b409-e131068fe430 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:04.959753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 806/1624] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:49:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:04.960953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dca6c-8512-401a-83e1-8987b408891c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:04.980663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043dca6c-8512-401a-83e1-8987b408891c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:04.981292 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 819/1625] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:04.982389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb56f66-53ec-4639-b409-e131068fe430 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:04.986879 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6f3ca-116f-41c5-bc6a-4e4500329d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:04.988894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6f3ca-116f-41c5-bc6a-4e4500329d2c admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:04.989254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6f3ca-116f-41c5-bc6a-4e4500329d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:04.989999 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6f3ca-116f-41c5-bc6a-4e4500329d2c admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:04.991023 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6f3ca-116f-41c5-bc6a-4e4500329d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:04.994755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6f3ca-116f-41c5-bc6a-4e4500329d2c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.012437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb56f66-53ec-4639-b409-e131068fe430 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.018203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b3fb4-22b2-4502-81f6-176540f885ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.019122 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b3fb4-22b2-4502-81f6-176540f885ad admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.019837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b3fb4-22b2-4502-81f6-176540f885ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.020324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b3fb4-22b2-4502-81f6-176540f885ad admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.021093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b3fb4-22b2-4502-81f6-176540f885ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.024616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b3fb4-22b2-4502-81f6-176540f885ad admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.034582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 807/1626] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:04 2022] POST /identity/v3/users => generated 309 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:05.041263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a3275-36ff-4758-a461-f686ac95d456 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.041676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a3275-36ff-4758-a461-f686ac95d456 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.041874 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a3275-36ff-4758-a461-f686ac95d456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.058419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 820/1627] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:04 2022] POST /identity/v3/users => generated 288 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:05.059493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b3fb4-22b2-4502-81f6-176540f885ad admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.086350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-800a3275-36ff-4758-a461-f686ac95d456 None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:05.087229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-800a3275-36ff-4758-a461-f686ac95d456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:05.105310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 808/1628] 10.222.0.20 () {58 vars in 1058 bytes} [Tue Jul 12 03:49:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:05.113192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a3275-36ff-4758-a461-f686ac95d456 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.136461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a3275-36ff-4758-a461-f686ac95d456 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.138046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b3fb4-22b2-4502-81f6-176540f885ad admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.143185 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f495bc-778e-47a2-b4e0-910baed41e0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.145010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f495bc-778e-47a2-b4e0-910baed41e0d service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.145010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f495bc-778e-47a2-b4e0-910baed41e0d service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.145010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f495bc-778e-47a2-b4e0-910baed41e0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.149002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc752dc-af07-4f93-af25-07749668c122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.149921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc752dc-af07-4f93-af25-07749668c122 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.150202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc752dc-af07-4f93-af25-07749668c122 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.150479 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc752dc-af07-4f93-af25-07749668c122 admin admin] PATH_INFO: `/v3/projects/9e3717707fa149efbf9d59296901cbd2/users/14d996c7e2124ad68232be116dd821f0/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.165280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f495bc-778e-47a2-b4e0-910baed41e0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.167498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f495bc-778e-47a2-b4e0-910baed41e0d service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.183717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 821/1629] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:05.184929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f495bc-778e-47a2-b4e0-910baed41e0d service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.189699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc752dc-af07-4f93-af25-07749668c122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e3717707fa149efbf9d59296901cbd2, user_id=14d996c7e2124ad68232be116dd821f0, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.192461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc752dc-af07-4f93-af25-07749668c122 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.197630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f495bc-778e-47a2-b4e0-910baed41e0d service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.206165 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a396d05-754a-45ca-ae09-60a0ddcc2eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.206964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396d05-754a-45ca-ae09-60a0ddcc2eee admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.207566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396d05-754a-45ca-ae09-60a0ddcc2eee admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.207985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396d05-754a-45ca-ae09-60a0ddcc2eee admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.208944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a396d05-754a-45ca-ae09-60a0ddcc2eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.212441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a396d05-754a-45ca-ae09-60a0ddcc2eee admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.220679 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 809/1630] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:05 2022] PUT /identity/v3/projects/9e3717707fa149efbf9d59296901cbd2/users/14d996c7e2124ad68232be116dd821f0/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:05.221606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc752dc-af07-4f93-af25-07749668c122 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.226754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 822/1631] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:05.227965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a396d05-754a-45ca-ae09-60a0ddcc2eee admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.246748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a396d05-754a-45ca-ae09-60a0ddcc2eee admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.350202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc752dc-af07-4f93-af25-07749668c122 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.359647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bc16f-d951-4e46-9ab0-926d38e5de3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.360372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bc16f-d951-4e46-9ab0-926d38e5de3c admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.360686 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bc16f-d951-4e46-9ab0-926d38e5de3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.361047 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bc16f-d951-4e46-9ab0-926d38e5de3c admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.361785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bc16f-d951-4e46-9ab0-926d38e5de3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.363618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bc16f-d951-4e46-9ab0-926d38e5de3c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.375511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b97f6-bb40-47fc-a2b4-6a2030f7f3d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.376217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b97f6-bb40-47fc-a2b4-6a2030f7f3d4 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.378800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b97f6-bb40-47fc-a2b4-6a2030f7f3d4 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.379124 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b97f6-bb40-47fc-a2b4-6a2030f7f3d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.388912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 810/1632] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:05.390144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bc16f-d951-4e46-9ab0-926d38e5de3c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.413497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bc16f-d951-4e46-9ab0-926d38e5de3c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.425874 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ed2b4-b023-4b9c-a239-4f1413bf9c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.426574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ed2b4-b023-4b9c-a239-4f1413bf9c58 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.426857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ed2b4-b023-4b9c-a239-4f1413bf9c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.427135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ed2b4-b023-4b9c-a239-4f1413bf9c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.427896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ed2b4-b023-4b9c-a239-4f1413bf9c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.429999 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ed2b4-b023-4b9c-a239-4f1413bf9c58 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.453784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 811/1633] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:05.457946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ed2b4-b023-4b9c-a239-4f1413bf9c58 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.477648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ed2b4-b023-4b9c-a239-4f1413bf9c58 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.479318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b97f6-bb40-47fc-a2b4-6a2030f7f3d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.482449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b97f6-bb40-47fc-a2b4-6a2030f7f3d4 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.490588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88cd26-c412-4b07-8202-d4caf4e94fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.491415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88cd26-c412-4b07-8202-d4caf4e94fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.491706 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88cd26-c412-4b07-8202-d4caf4e94fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.492037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88cd26-c412-4b07-8202-d4caf4e94fb0 admin admin] PATH_INFO: `/v3/projects/9ad9b5c582f74968840c5aeb13fcbfe8/users/82486ce65dab4b9faef6188d4f43e218/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.501800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 823/1634] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:05.504178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b97f6-bb40-47fc-a2b4-6a2030f7f3d4 service nova] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.510555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:05.519063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b97f6-bb40-47fc-a2b4-6a2030f7f3d4 service nova] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.537241 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfcc23-7529-4d99-ba6e-51fe3d715dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.538055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfcc23-7529-4d99-ba6e-51fe3d715dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.538404 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfcc23-7529-4d99-ba6e-51fe3d715dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.539471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfcc23-7529-4d99-ba6e-51fe3d715dc8 admin admin] PATH_INFO: `/v3/projects/dee515d5612e4524963a0f2c38791419/users/deab1fc9cdfb40d7928ac2ea0e6c6003/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.549049 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88cd26-c412-4b07-8202-d4caf4e94fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ad9b5c582f74968840c5aeb13fcbfe8, user_id=82486ce65dab4b9faef6188d4f43e218, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.552217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88cd26-c412-4b07-8202-d4caf4e94fb0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.568081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfcc23-7529-4d99-ba6e-51fe3d715dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee515d5612e4524963a0f2c38791419, user_id=deab1fc9cdfb40d7928ac2ea0e6c6003, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.571322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfcc23-7529-4d99-ba6e-51fe3d715dc8 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.591777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 812/1635] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:05 2022] PUT /identity/v3/projects/9ad9b5c582f74968840c5aeb13fcbfe8/users/82486ce65dab4b9faef6188d4f43e218/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:05.592681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88cd26-c412-4b07-8202-d4caf4e94fb0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.603845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 824/1636] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:05 2022] PUT /identity/v3/projects/dee515d5612e4524963a0f2c38791419/users/deab1fc9cdfb40d7928ac2ea0e6c6003/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:05.603845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfcc23-7529-4d99-ba6e-51fe3d715dc8 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.716896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfcc23-7529-4d99-ba6e-51fe3d715dc8 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.723668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c0cfe-1049-4907-8df0-256ef3b737ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.724700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c0cfe-1049-4907-8df0-256ef3b737ee admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.725236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c0cfe-1049-4907-8df0-256ef3b737ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.726049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c0cfe-1049-4907-8df0-256ef3b737ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.727120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c0cfe-1049-4907-8df0-256ef3b737ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.729290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c0cfe-1049-4907-8df0-256ef3b737ee admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.747663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 825/1637] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:05.749129 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c0cfe-1049-4907-8df0-256ef3b737ee admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.762445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c0cfe-1049-4907-8df0-256ef3b737ee admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.765092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88cd26-c412-4b07-8202-d4caf4e94fb0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.768703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03916004-8fec-4dcc-95df-0340cbdf01d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.769739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03916004-8fec-4dcc-95df-0340cbdf01d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.770292 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03916004-8fec-4dcc-95df-0340cbdf01d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.770968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03916004-8fec-4dcc-95df-0340cbdf01d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.771911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03916004-8fec-4dcc-95df-0340cbdf01d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.774718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03916004-8fec-4dcc-95df-0340cbdf01d3 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.779740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50763d21-4ad2-4a0d-adf6-96fe2a74e927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.780972 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50763d21-4ad2-4a0d-adf6-96fe2a74e927 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.781268 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50763d21-4ad2-4a0d-adf6-96fe2a74e927 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.781597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50763d21-4ad2-4a0d-adf6-96fe2a74e927 admin admin] PATH_INFO: `/v3/projects/9e3717707fa149efbf9d59296901cbd2/users/14d996c7e2124ad68232be116dd821f0/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.786106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50763d21-4ad2-4a0d-adf6-96fe2a74e927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e3717707fa149efbf9d59296901cbd2, user_id=14d996c7e2124ad68232be116dd821f0, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.790352 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 826/1638] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:05.791681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03916004-8fec-4dcc-95df-0340cbdf01d3 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.793384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50763d21-4ad2-4a0d-adf6-96fe2a74e927 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.798785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:05.813436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03916004-8fec-4dcc-95df-0340cbdf01d3 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:05.825409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 813/1639] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:05 2022] PUT /identity/v3/projects/9e3717707fa149efbf9d59296901cbd2/users/14d996c7e2124ad68232be116dd821f0/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:05.826920 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50763d21-4ad2-4a0d-adf6-96fe2a74e927 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:05.947746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098c8bb-e4bd-4744-842a-b37b34f88817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:05.948706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098c8bb-e4bd-4744-842a-b37b34f88817 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.948996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098c8bb-e4bd-4744-842a-b37b34f88817 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.949274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098c8bb-e4bd-4744-842a-b37b34f88817 admin admin] PATH_INFO: `/v3/projects/9ad9b5c582f74968840c5aeb13fcbfe8/users/82486ce65dab4b9faef6188d4f43e218/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:05.953667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098c8bb-e4bd-4744-842a-b37b34f88817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ad9b5c582f74968840c5aeb13fcbfe8, user_id=82486ce65dab4b9faef6188d4f43e218, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:05.956426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098c8bb-e4bd-4744-842a-b37b34f88817 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:05.988582 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 827/1640] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:05 2022] PUT /identity/v3/projects/9ad9b5c582f74968840c5aeb13fcbfe8/users/82486ce65dab4b9faef6188d4f43e218/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:05.989685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b792994-1af9-4f15-9af4-ef873c1e5b54 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:05.989978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b792994-1af9-4f15-9af4-ef873c1e5b54 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:05.990259 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b792994-1af9-4f15-9af4-ef873c1e5b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:06.001015 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50763d21-4ad2-4a0d-adf6-96fe2a74e927 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:06.074698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-4b792994-1af9-4f15-9af4-ef873c1e5b54 None None] MFA Rules not processed for user `14d996c7e2124ad68232be116dd821f0`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:06.078479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4b792994-1af9-4f15-9af4-ef873c1e5b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:06.123360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef20cf3-bbb9-4eae-ac1e-279d70da5f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:06.125080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef20cf3-bbb9-4eae-ac1e-279d70da5f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:06.125080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef20cf3-bbb9-4eae-ac1e-279d70da5f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:06.125580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef20cf3-bbb9-4eae-ac1e-279d70da5f25 admin admin] PATH_INFO: `/v3/projects/dee515d5612e4524963a0f2c38791419/users/deab1fc9cdfb40d7928ac2ea0e6c6003/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:06.127949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef20cf3-bbb9-4eae-ac1e-279d70da5f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee515d5612e4524963a0f2c38791419, user_id=deab1fc9cdfb40d7928ac2ea0e6c6003, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:06.131147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef20cf3-bbb9-4eae-ac1e-279d70da5f25 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:06.152856 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 814/1641] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:05 2022] PUT /identity/v3/projects/dee515d5612e4524963a0f2c38791419/users/deab1fc9cdfb40d7928ac2ea0e6c6003/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:06.162529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de843d29-da3b-4e67-b141-ca67e7b97ad7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:06.162529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de843d29-da3b-4e67-b141-ca67e7b97ad7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:06.162529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de843d29-da3b-4e67-b141-ca67e7b97ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:06.181286 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:06.224955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-de843d29-da3b-4e67-b141-ca67e7b97ad7 None None] MFA Rules not processed for user `82486ce65dab4b9faef6188d4f43e218`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:06.227773 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-de843d29-da3b-4e67-b141-ca67e7b97ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:06.244241 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 828/1642] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:05 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:06.246149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c530d6-90d8-48fe-aab4-cf0a658a0c5d None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:06.251910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c530d6-90d8-48fe-aab4-cf0a658a0c5d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:06.252201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c530d6-90d8-48fe-aab4-cf0a658a0c5d None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:06.257068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 829/1643] 10.222.0.20 () {54 vars in 912 bytes} [Tue Jul 12 03:49:06 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:49:06.260387 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6416c2-89f1-4e33-b941-4b8e494d559f None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:06.260709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6416c2-89f1-4e33-b941-4b8e494d559f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:06.260987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6416c2-89f1-4e33-b941-4b8e494d559f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:06.329079 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-6f6416c2-89f1-4e33-b941-4b8e494d559f None None] MFA Rules not processed for user `deab1fc9cdfb40d7928ac2ea0e6c6003`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:06.330333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6f6416c2-89f1-4e33-b941-4b8e494d559f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:06.465747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 830/1644] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:06 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:06.466780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6416c2-89f1-4e33-b941-4b8e494d559f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:06.468843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 815/1645] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:06 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:06.470625 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9b7a9-34ab-4f79-9e8b-9d7ba457b3bd None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:06.471117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9b7a9-34ab-4f79-9e8b-9d7ba457b3bd None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:06.471599 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9b7a9-34ab-4f79-9e8b-9d7ba457b3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:06.519135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-f9a9b7a9-34ab-4f79-9e8b-9d7ba457b3bd None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:06.520719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f9a9b7a9-34ab-4f79-9e8b-9d7ba457b3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:06.551797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:06.584428 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6416c2-89f1-4e33-b941-4b8e494d559f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:06.599573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7ce56-d611-47f6-aa77-f32eb9ff1bbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:06.600998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7ce56-d611-47f6-aa77-f32eb9ff1bbd service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:06.601348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7ce56-d611-47f6-aa77-f32eb9ff1bbd service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:06.601633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7ce56-d611-47f6-aa77-f32eb9ff1bbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:06.644922 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 816/1646] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:49:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:06.645882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9b7a9-34ab-4f79-9e8b-9d7ba457b3bd None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:06.673970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9b7a9-34ab-4f79-9e8b-9d7ba457b3bd None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:06.683375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed812b1b-65d4-4a5e-9d5d-0819cca1a1ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:06.686145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed812b1b-65d4-4a5e-9d5d-0819cca1a1ad service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:06.686439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed812b1b-65d4-4a5e-9d5d-0819cca1a1ad service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:06.686717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed812b1b-65d4-4a5e-9d5d-0819cca1a1ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:06.715982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed812b1b-65d4-4a5e-9d5d-0819cca1a1ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:06.720985 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed812b1b-65d4-4a5e-9d5d-0819cca1a1ad service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:06.759775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 817/1647] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:06 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:06.791968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7ce56-d611-47f6-aa77-f32eb9ff1bbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:06.795634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7ce56-d611-47f6-aa77-f32eb9ff1bbd service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:06.815856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 831/1648] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:06 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:06.837813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed812b1b-65d4-4a5e-9d5d-0819cca1a1ad service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:06.849087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed812b1b-65d4-4a5e-9d5d-0819cca1a1ad service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:06.856874 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b47de-6470-44f4-8b16-799be1fe1753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:06.858659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b47de-6470-44f4-8b16-799be1fe1753 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:06.859255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b47de-6470-44f4-8b16-799be1fe1753 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:06.859403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b47de-6470-44f4-8b16-799be1fe1753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:06.872137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b47de-6470-44f4-8b16-799be1fe1753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:06.873944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b47de-6470-44f4-8b16-799be1fe1753 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:06.897203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7ce56-d611-47f6-aa77-f32eb9ff1bbd service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:06.920092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 818/1649] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:06 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:06.923911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7ce56-d611-47f6-aa77-f32eb9ff1bbd service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:06.928599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88b1da-9baf-426a-a776-91ecfe6b7b5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:06.929421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88b1da-9baf-426a-a776-91ecfe6b7b5b service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:06.929963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88b1da-9baf-426a-a776-91ecfe6b7b5b service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:06.930375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88b1da-9baf-426a-a776-91ecfe6b7b5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:06.975115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88b1da-9baf-426a-a776-91ecfe6b7b5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:06.978246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88b1da-9baf-426a-a776-91ecfe6b7b5b service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:07.010475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 832/1650] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:49:07.225898 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b47de-6470-44f4-8b16-799be1fe1753 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:07.238092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b47de-6470-44f4-8b16-799be1fe1753 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:07.246493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db2b64-2a92-493b-bb4f-3065a642cb38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:07.248872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db2b64-2a92-493b-bb4f-3065a642cb38 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:07.250379 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db2b64-2a92-493b-bb4f-3065a642cb38 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:07.250997 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db2b64-2a92-493b-bb4f-3065a642cb38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:07.419124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db2b64-2a92-493b-bb4f-3065a642cb38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:07.422346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db2b64-2a92-493b-bb4f-3065a642cb38 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:07.446862 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 819/1651] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:07 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:07.477512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88b1da-9baf-426a-a776-91ecfe6b7b5b service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:07.483533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:07.487326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88b1da-9baf-426a-a776-91ecfe6b7b5b service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:07.497127 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f30f93-8fbb-4527-8413-18b5c04201af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:07.498371 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f30f93-8fbb-4527-8413-18b5c04201af service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:07.498683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f30f93-8fbb-4527-8413-18b5c04201af service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:07.498958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f30f93-8fbb-4527-8413-18b5c04201af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:07.692093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f30f93-8fbb-4527-8413-18b5c04201af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:07.693591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f30f93-8fbb-4527-8413-18b5c04201af service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:07.739108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 833/1652] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:07 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:08.186529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db2b64-2a92-493b-bb4f-3065a642cb38 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:08.363742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db2b64-2a92-493b-bb4f-3065a642cb38 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:08.380210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:08.381837 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:08.383346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:08.384317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] PATH_INFO: `/v3/users/17a48195a14a40c4b4e6420636252783` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:08.387017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17a48195a14a40c4b4e6420636252783)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:08.390380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:08.481626 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:08.542628 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17a48195a14a40c4b4e6420636252783'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:08.558285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17a48195a14a40c4b4e6420636252783'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:08.559404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17a48195a14a40c4b4e6420636252783'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:08.593390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17a48195a14a40c4b4e6420636252783'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:08.640071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17a48195a14a40c4b4e6420636252783'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:08.657451 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 820/1653] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:08 2022] DELETE /identity/v3/users/17a48195a14a40c4b4e6420636252783 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:08.659062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a759cbb-9940-4dad-9217-1eb36692ae92 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:08.659577 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a759cbb-9940-4dad-9217-1eb36692ae92 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:08.659680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a759cbb-9940-4dad-9217-1eb36692ae92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:08.701977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-1a759cbb-9940-4dad-9217-1eb36692ae92 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:08.704108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1a759cbb-9940-4dad-9217-1eb36692ae92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:08.904616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 834/1654] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:08.904894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:08.904894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:09.044094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef52acc-513a-4ebe-994c-f108094b88c6 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:09.048968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9371f-29df-4df8-a617-73b1acdb297e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:09.049606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9371f-29df-4df8-a617-73b1acdb297e service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:09.049880 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9371f-29df-4df8-a617-73b1acdb297e service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:09.050560 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9371f-29df-4df8-a617-73b1acdb297e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:09.061442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9371f-29df-4df8-a617-73b1acdb297e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:09.062942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9371f-29df-4df8-a617-73b1acdb297e service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:09.081821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 821/1655] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:10.993505 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a759cbb-9940-4dad-9217-1eb36692ae92 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:10.996801 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1a759cbb-9940-4dad-9217-1eb36692ae92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:11.019024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a759cbb-9940-4dad-9217-1eb36692ae92 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:11.030982 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:11.032080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:11.034675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:11.035204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] PATH_INFO: `/v3/projects/dcd9e1e7c72b4f6fa85068bce0cf35eb` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:11.039250 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcd9e1e7c72b4f6fa85068bce0cf35eb)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:11.052401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:11.181944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcd9e1e7c72b4f6fa85068bce0cf35eb', 'request_id': 'req-20b1771b-ed86-4e02-aa56-304697ad12cc'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:11.215920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcd9e1e7c72b4f6fa85068bce0cf35eb', 'request_id': 'req-20b1771b-ed86-4e02-aa56-304697ad12cc'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:11.228511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcd9e1e7c72b4f6fa85068bce0cf35eb', 'request_id': 'req-20b1771b-ed86-4e02-aa56-304697ad12cc'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:11.268091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] The token cache is being invalidate because project dcd9e1e7c72b4f6fa85068bce0cf35eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:11.268754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:11.270960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:11.288380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 835/1656] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:10 2022] DELETE /identity/v3/projects/dcd9e1e7c72b4f6fa85068bce0cf35eb => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:11.293796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9371f-29df-4df8-a617-73b1acdb297e service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:11.317684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9371f-29df-4df8-a617-73b1acdb297e service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:11.540569 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2e1dc-b537-408d-9896-354da1b73600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:11.542125 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2e1dc-b537-408d-9896-354da1b73600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:11.542349 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2e1dc-b537-408d-9896-354da1b73600 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:11.542680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2e1dc-b537-408d-9896-354da1b73600 admin admin] PATH_INFO: `/v3/users/82486ce65dab4b9faef6188d4f43e218` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:11.550073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2e1dc-b537-408d-9896-354da1b73600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82486ce65dab4b9faef6188d4f43e218)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:11.552183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2e1dc-b537-408d-9896-354da1b73600 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:11.709446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3fc2e1dc-b537-408d-9896-354da1b73600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82486ce65dab4b9faef6188d4f43e218'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:11.712309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:11.731749 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3fc2e1dc-b537-408d-9896-354da1b73600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82486ce65dab4b9faef6188d4f43e218'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:11.732955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3fc2e1dc-b537-408d-9896-354da1b73600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82486ce65dab4b9faef6188d4f43e218'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:11.745334 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3fc2e1dc-b537-408d-9896-354da1b73600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82486ce65dab4b9faef6188d4f43e218'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:11.795194 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3fc2e1dc-b537-408d-9896-354da1b73600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82486ce65dab4b9faef6188d4f43e218'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:11.832250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 822/1657] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:11 2022] DELETE /identity/v3/users/82486ce65dab4b9faef6188d4f43e218 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:12.839748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:12.841823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:12.876844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1771b-ed86-4e02-aa56-304697ad12cc admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:12.957718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:12.998549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e54bc-c0ec-4be4-975d-181f3c32f263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:12.999724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e54bc-c0ec-4be4-975d-181f3c32f263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:13.000033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e54bc-c0ec-4be4-975d-181f3c32f263 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:13.000311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e54bc-c0ec-4be4-975d-181f3c32f263 admin admin] PATH_INFO: `/v3/projects/9ad9b5c582f74968840c5aeb13fcbfe8` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:13.001648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e54bc-c0ec-4be4-975d-181f3c32f263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ad9b5c582f74968840c5aeb13fcbfe8)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:13.005049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e54bc-c0ec-4be4-975d-181f3c32f263 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:13.093969 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ab3e54bc-c0ec-4be4-975d-181f3c32f263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ad9b5c582f74968840c5aeb13fcbfe8', 'request_id': 'req-ab3e54bc-c0ec-4be4-975d-181f3c32f263'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:13.108462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ab3e54bc-c0ec-4be4-975d-181f3c32f263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ad9b5c582f74968840c5aeb13fcbfe8', 'request_id': 'req-ab3e54bc-c0ec-4be4-975d-181f3c32f263'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:13.120378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ab3e54bc-c0ec-4be4-975d-181f3c32f263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ad9b5c582f74968840c5aeb13fcbfe8', 'request_id': 'req-ab3e54bc-c0ec-4be4-975d-181f3c32f263'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:13.136504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ab3e54bc-c0ec-4be4-975d-181f3c32f263 admin admin] The token cache is being invalidate because project 9ad9b5c582f74968840c5aeb13fcbfe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:13.136933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ab3e54bc-c0ec-4be4-975d-181f3c32f263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:13.137672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ab3e54bc-c0ec-4be4-975d-181f3c32f263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:13.143393 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 836/1658] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:12 2022] DELETE /identity/v3/projects/9ad9b5c582f74968840c5aeb13fcbfe8 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:13.145014 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47457e-856c-4325-b989-9f2f0da5a4e9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:13.145786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47457e-856c-4325-b989-9f2f0da5a4e9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:13.146196 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47457e-856c-4325-b989-9f2f0da5a4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:13.203441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2e47457e-856c-4325-b989-9f2f0da5a4e9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:13.205357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2e47457e-856c-4325-b989-9f2f0da5a4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:13.357484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 823/1659] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:13.365610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10284e-5307-4fe6-a200-66c0bb0f3aae None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:13.366320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10284e-5307-4fe6-a200-66c0bb0f3aae None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:13.366559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10284e-5307-4fe6-a200-66c0bb0f3aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:13.411003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-ae10284e-5307-4fe6-a200-66c0bb0f3aae None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:13.415710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ae10284e-5307-4fe6-a200-66c0bb0f3aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:13.439711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 837/1660] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:13.445356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47457e-856c-4325-b989-9f2f0da5a4e9 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:13.460057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47457e-856c-4325-b989-9f2f0da5a4e9 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:13.481721 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880af992-d8de-4cb6-b485-4d7f0a891b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:13.482736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880af992-d8de-4cb6-b485-4d7f0a891b5d admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:13.482911 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880af992-d8de-4cb6-b485-4d7f0a891b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:13.483225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880af992-d8de-4cb6-b485-4d7f0a891b5d admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:13.483974 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880af992-d8de-4cb6-b485-4d7f0a891b5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:13.486977 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880af992-d8de-4cb6-b485-4d7f0a891b5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:13.489323 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880af992-d8de-4cb6-b485-4d7f0a891b5d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:13.533998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 824/1661] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:13.539049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10284e-5307-4fe6-a200-66c0bb0f3aae None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:13.549813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10284e-5307-4fe6-a200-66c0bb0f3aae None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:13.556151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7567c-4f9f-4201-953c-c08fbdfc85ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:13.556842 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7567c-4f9f-4201-953c-c08fbdfc85ba admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:13.557130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7567c-4f9f-4201-953c-c08fbdfc85ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:13.557410 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7567c-4f9f-4201-953c-c08fbdfc85ba admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:13.558187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7567c-4f9f-4201-953c-c08fbdfc85ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:13.561036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7567c-4f9f-4201-953c-c08fbdfc85ba admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:13.598678 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7ad9e-52f6-46c0-afd7-208006e0c82b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:13.599506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7ad9e-52f6-46c0-afd7-208006e0c82b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:13.599948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7ad9e-52f6-46c0-afd7-208006e0c82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:13.616170 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 838/1662] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:13 2022] POST /identity/v3/projects => generated 363 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:13.617829 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7567c-4f9f-4201-953c-c08fbdfc85ba admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:13.675674 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9fb7ad9e-52f6-46c0-afd7-208006e0c82b None None] MFA Rules not processed for user `4ac2825db0c649cab6dcc2e27fbaf80d`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:13.679835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9fb7ad9e-52f6-46c0-afd7-208006e0c82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:13.730131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7567c-4f9f-4201-953c-c08fbdfc85ba admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:13.735459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d017fc-9bab-4292-8c00-10f63c2d8e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:13.736703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d017fc-9bab-4292-8c00-10f63c2d8e18 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:13.737051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d017fc-9bab-4292-8c00-10f63c2d8e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:13.737343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d017fc-9bab-4292-8c00-10f63c2d8e18 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:13.738299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d017fc-9bab-4292-8c00-10f63c2d8e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:13.740885 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d017fc-9bab-4292-8c00-10f63c2d8e18 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:13.782998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 839/1663] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:13 2022] POST /identity/v3/users => generated 282 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:13.785140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721732fd-277d-4979-9ffc-89827229c877 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:13.785439 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721732fd-277d-4979-9ffc-89827229c877 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:13.785769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721732fd-277d-4979-9ffc-89827229c877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:13.835037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-721732fd-277d-4979-9ffc-89827229c877 None None] MFA Rules not processed for user `70ea64f67f264b20a7c556c625a64f80`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:13.836560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-721732fd-277d-4979-9ffc-89827229c877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:13.843431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:13.881616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 825/1664] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:13.881616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7ad9e-52f6-46c0-afd7-208006e0c82b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:13.898934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7ad9e-52f6-46c0-afd7-208006e0c82b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:13.912813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73108c-b6f4-41d1-ba8b-645abc7d25fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:13.914595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73108c-b6f4-41d1-ba8b-645abc7d25fc admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:13.914595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73108c-b6f4-41d1-ba8b-645abc7d25fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:13.914595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73108c-b6f4-41d1-ba8b-645abc7d25fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:13.915299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73108c-b6f4-41d1-ba8b-645abc7d25fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:13.916905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73108c-b6f4-41d1-ba8b-645abc7d25fc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:13.938051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 826/1665] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:13.938051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7360293-ccab-468d-9cce-5861254f66e8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:13.938051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7360293-ccab-468d-9cce-5861254f66e8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:13.938051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7360293-ccab-468d-9cce-5861254f66e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:14.000772 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b7360293-ccab-468d-9cce-5861254f66e8 None None] MFA Rules not processed for user `4ac2825db0c649cab6dcc2e27fbaf80d`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:14.005376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b7360293-ccab-468d-9cce-5861254f66e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:14.028778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 827/1666] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:14.033460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7360293-ccab-468d-9cce-5861254f66e8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:14.054743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 840/1667] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:14.057128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add9e22-abe3-4c58-9f13-646a7952571c None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:14.057434 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add9e22-abe3-4c58-9f13-646a7952571c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:14.057718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add9e22-abe3-4c58-9f13-646a7952571c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:14.092182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7360293-ccab-468d-9cce-5861254f66e8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:14.097673 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-0add9e22-abe3-4c58-9f13-646a7952571c None None] MFA Rules not processed for user `70ea64f67f264b20a7c556c625a64f80`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:14.099050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0add9e22-abe3-4c58-9f13-646a7952571c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:14.099243 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc116dc-790e-4eb1-9d7a-194a93dc85ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:14.099965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc116dc-790e-4eb1-9d7a-194a93dc85ff admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:14.100361 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc116dc-790e-4eb1-9d7a-194a93dc85ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:14.100688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc116dc-790e-4eb1-9d7a-194a93dc85ff admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:14.101428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc116dc-790e-4eb1-9d7a-194a93dc85ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:14.103448 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc116dc-790e-4eb1-9d7a-194a93dc85ff admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:14.121969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 828/1668] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:14.122939 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc116dc-790e-4eb1-9d7a-194a93dc85ff admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:14.125134 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ccc116dc-790e-4eb1-9d7a-194a93dc85ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:14.128804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 841/1669] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:14.135350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add9e22-abe3-4c58-9f13-646a7952571c None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:14.157634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:14.173193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add9e22-abe3-4c58-9f13-646a7952571c None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:14.184733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5a2e1-7ab5-4d94-8594-0635dc65e5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:14.185671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5a2e1-7ab5-4d94-8594-0635dc65e5bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:14.185957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5a2e1-7ab5-4d94-8594-0635dc65e5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:14.186243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5a2e1-7ab5-4d94-8594-0635dc65e5bd admin admin] PATH_INFO: `/v3/projects/81be5c4d248246a28ca48d7448a4ca9f/users/6722ba26f2fc405385b7713cdbc95cdc/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:14.243814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5a2e1-7ab5-4d94-8594-0635dc65e5bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=81be5c4d248246a28ca48d7448a4ca9f, user_id=6722ba26f2fc405385b7713cdbc95cdc, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:14.257904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5a2e1-7ab5-4d94-8594-0635dc65e5bd admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:14.305481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 842/1670] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:14 2022] PUT /identity/v3/projects/81be5c4d248246a28ca48d7448a4ca9f/users/6722ba26f2fc405385b7713cdbc95cdc/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:14.305481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5a2e1-7ab5-4d94-8594-0635dc65e5bd admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:14.305481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-02c5a2e1-7ab5-4d94-8594-0635dc65e5bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:14.374832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc116dc-790e-4eb1-9d7a-194a93dc85ff admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:14.462267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6b0c7-0f45-432a-a12f-cba36f03ac72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:14.462968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6b0c7-0f45-432a-a12f-cba36f03ac72 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:14.463258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6b0c7-0f45-432a-a12f-cba36f03ac72 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:14.463538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6b0c7-0f45-432a-a12f-cba36f03ac72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:14.540053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6b0c7-0f45-432a-a12f-cba36f03ac72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:14.541857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6b0c7-0f45-432a-a12f-cba36f03ac72 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:14.566832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5a2e1-7ab5-4d94-8594-0635dc65e5bd admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:14.576620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 829/1671] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:14.576859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa175a-7f4d-4014-8027-6b3593e42f4d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:14.577180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa175a-7f4d-4014-8027-6b3593e42f4d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:14.577418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa175a-7f4d-4014-8027-6b3593e42f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:14.592588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cd090-c933-44d7-8f77-fc140cba708f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:14.593777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cd090-c933-44d7-8f77-fc140cba708f service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:14.594291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cd090-c933-44d7-8f77-fc140cba708f service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:14.594923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cd090-c933-44d7-8f77-fc140cba708f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:14.684070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-18fa175a-7f4d-4014-8027-6b3593e42f4d None None] MFA Rules not processed for user `6722ba26f2fc405385b7713cdbc95cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:14.685285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-18fa175a-7f4d-4014-8027-6b3593e42f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:14.706048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cd090-c933-44d7-8f77-fc140cba708f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:14.707674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cd090-c933-44d7-8f77-fc140cba708f service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:14.761758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:14.773786 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 843/1672] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:14.908805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 830/1673] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:14 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:14.913653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cd090-c933-44d7-8f77-fc140cba708f service nova] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:14.940770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:15.064897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cd090-c933-44d7-8f77-fc140cba708f service nova] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:15.072191 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63d044-0feb-4a47-8524-ee31c02a340d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:15.072886 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d044-0feb-4a47-8524-ee31c02a340d admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:15.073170 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d044-0feb-4a47-8524-ee31c02a340d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:15.073444 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d044-0feb-4a47-8524-ee31c02a340d admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:15.074148 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63d044-0feb-4a47-8524-ee31c02a340d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:15.076132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63d044-0feb-4a47-8524-ee31c02a340d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:15.127647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 844/1674] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:14 2022] POST /identity/v3/projects => generated 365 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:15.131710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa175a-7f4d-4014-8027-6b3593e42f4d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:15.148895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa175a-7f4d-4014-8027-6b3593e42f4d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:15.252429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be3502-5e51-4cb9-915f-429966b2eb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:15.253095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be3502-5e51-4cb9-915f-429966b2eb6c admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:15.253392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be3502-5e51-4cb9-915f-429966b2eb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:15.253670 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be3502-5e51-4cb9-915f-429966b2eb6c admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:15.254597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be3502-5e51-4cb9-915f-429966b2eb6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:15.264660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be3502-5e51-4cb9-915f-429966b2eb6c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:15.317155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 831/1675] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:15 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:15.318099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63d044-0feb-4a47-8524-ee31c02a340d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:15.336538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63d044-0feb-4a47-8524-ee31c02a340d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:15.357480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c059acc-eb1a-46b9-a6b8-873018c41f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:15.372588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c059acc-eb1a-46b9-a6b8-873018c41f2f admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:15.375464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c059acc-eb1a-46b9-a6b8-873018c41f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:15.375968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c059acc-eb1a-46b9-a6b8-873018c41f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:15.379164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c059acc-eb1a-46b9-a6b8-873018c41f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:15.386357 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c059acc-eb1a-46b9-a6b8-873018c41f2f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:15.420374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 845/1676] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:15.422329 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be3502-5e51-4cb9-915f-429966b2eb6c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:15.439460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be3502-5e51-4cb9-915f-429966b2eb6c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:15.452944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19e6ae-03e8-420e-a910-0f5300208590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:15.455432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19e6ae-03e8-420e-a910-0f5300208590 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:15.455972 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19e6ae-03e8-420e-a910-0f5300208590 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:15.456905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19e6ae-03e8-420e-a910-0f5300208590 admin admin] PATH_INFO: `/v3/projects/e61b7267acd6432782785446abcb706a/users/24423b12130f416ab15d32c4f2e854d1/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:15.505335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19e6ae-03e8-420e-a910-0f5300208590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e61b7267acd6432782785446abcb706a, user_id=24423b12130f416ab15d32c4f2e854d1, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:15.511679 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19e6ae-03e8-420e-a910-0f5300208590 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:15.556865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 832/1677] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:15 2022] PUT /identity/v3/projects/e61b7267acd6432782785446abcb706a/users/24423b12130f416ab15d32c4f2e854d1/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:15.558500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c059acc-eb1a-46b9-a6b8-873018c41f2f admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:15.575617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c059acc-eb1a-46b9-a6b8-873018c41f2f admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:15.768856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0a1ac-2364-4052-a074-a222f95ef348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:15.769929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0a1ac-2364-4052-a074-a222f95ef348 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:15.770394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0a1ac-2364-4052-a074-a222f95ef348 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:15.770849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0a1ac-2364-4052-a074-a222f95ef348 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:15.771759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0a1ac-2364-4052-a074-a222f95ef348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:15.774437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0a1ac-2364-4052-a074-a222f95ef348 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:15.794736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19e6ae-03e8-420e-a910-0f5300208590 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:15.807295 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19e6ae-03e8-420e-a910-0f5300208590 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:15.808589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 846/1678] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:15.817179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4c1b4-1e29-45fe-a3ab-ee342efad958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:15.818246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4c1b4-1e29-45fe-a3ab-ee342efad958 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:15.818941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4c1b4-1e29-45fe-a3ab-ee342efad958 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:15.819102 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4c1b4-1e29-45fe-a3ab-ee342efad958 admin admin] PATH_INFO: `/v3/projects/e61b7267acd6432782785446abcb706a/users/24423b12130f416ab15d32c4f2e854d1/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:15.821428 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4c1b4-1e29-45fe-a3ab-ee342efad958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e61b7267acd6432782785446abcb706a, user_id=24423b12130f416ab15d32c4f2e854d1, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:15.824849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4c1b4-1e29-45fe-a3ab-ee342efad958 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:15.856552 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d4c9c-2793-4c96-a525-682114527afb None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:15.856935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d4c9c-2793-4c96-a525-682114527afb None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:15.857287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d4c9c-2793-4c96-a525-682114527afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:15.859169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 833/1679] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:15 2022] PUT /identity/v3/projects/e61b7267acd6432782785446abcb706a/users/24423b12130f416ab15d32c4f2e854d1/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:15.927181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-fc4d4c9c-2793-4c96-a525-682114527afb None None] MFA Rules not processed for user `24423b12130f416ab15d32c4f2e854d1`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:15.930237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-fc4d4c9c-2793-4c96-a525-682114527afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:15.989012 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711c28c-f794-4568-a9cf-60bfeab27295 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:15.989423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711c28c-f794-4568-a9cf-60bfeab27295 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:15.989828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711c28c-f794-4568-a9cf-60bfeab27295 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:15.991733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 834/1680] 10.222.0.20 () {54 vars in 912 bytes} [Tue Jul 12 03:49:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:49:16.005113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fa156-0ec0-44aa-a122-08b38d60587b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:16.006295 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fa156-0ec0-44aa-a122-08b38d60587b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:16.007113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fa156-0ec0-44aa-a122-08b38d60587b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:16.107096 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-293fa156-0ec0-44aa-a122-08b38d60587b None None] MFA Rules not processed for user `139be21e7f644b5796d30158851ebd00`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:16.107096 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-293fa156-0ec0-44aa-a122-08b38d60587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:16.161928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 847/1681] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:16.163574 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d4d32-18e8-42ec-87f6-e01abb82d929 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:16.163866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d4d32-18e8-42ec-87f6-e01abb82d929 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:16.164654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d4d32-18e8-42ec-87f6-e01abb82d929 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:16.166577 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 848/1682] 10.222.0.20 () {54 vars in 912 bytes} [Tue Jul 12 03:49:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:49:16.174699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9586ec-b270-4bb1-bd47-6d19343db684 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:16.175256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9586ec-b270-4bb1-bd47-6d19343db684 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:16.175675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9586ec-b270-4bb1-bd47-6d19343db684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:16.226585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2e9586ec-b270-4bb1-bd47-6d19343db684 None None] MFA Rules not processed for user `139be21e7f644b5796d30158851ebd00`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:16.227827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2e9586ec-b270-4bb1-bd47-6d19343db684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:16.240335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:16.281452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 835/1683] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:16.286518 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fa156-0ec0-44aa-a122-08b38d60587b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:16.289412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-293fa156-0ec0-44aa-a122-08b38d60587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:16.361120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 849/1684] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:16.370369 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:16.375923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9586ec-b270-4bb1-bd47-6d19343db684 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:16.377534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2e9586ec-b270-4bb1-bd47-6d19343db684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:16.395475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9586ec-b270-4bb1-bd47-6d19343db684 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:16.406184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5349b-c973-4fd3-87fa-942379c88b9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:16.406899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5349b-c973-4fd3-87fa-942379c88b9b service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:16.408301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5349b-c973-4fd3-87fa-942379c88b9b service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:16.408858 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5349b-c973-4fd3-87fa-942379c88b9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:16.423364 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fa156-0ec0-44aa-a122-08b38d60587b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:16.430360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34775fad-7448-413f-96dd-5b9fd64c8a78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:16.431401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34775fad-7448-413f-96dd-5b9fd64c8a78 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:16.431901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34775fad-7448-413f-96dd-5b9fd64c8a78 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:16.432408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34775fad-7448-413f-96dd-5b9fd64c8a78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:16.436024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5349b-c973-4fd3-87fa-942379c88b9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:16.437833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5349b-c973-4fd3-87fa-942379c88b9b service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:16.452587 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 850/1685] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:49:16.453451 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5349b-c973-4fd3-87fa-942379c88b9b service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:16.454993 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6ac5349b-c973-4fd3-87fa-942379c88b9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:16.472663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34775fad-7448-413f-96dd-5b9fd64c8a78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:16.475095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34775fad-7448-413f-96dd-5b9fd64c8a78 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:16.482254 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5349b-c973-4fd3-87fa-942379c88b9b service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:16.491076 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7ed48-e285-47ef-b95c-983ad74bbbc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:16.491758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7ed48-e285-47ef-b95c-983ad74bbbc8 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:16.492072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7ed48-e285-47ef-b95c-983ad74bbbc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:16.492348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7ed48-e285-47ef-b95c-983ad74bbbc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:16.513560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7ed48-e285-47ef-b95c-983ad74bbbc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:16.515055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7ed48-e285-47ef-b95c-983ad74bbbc8 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:16.533719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 851/1686] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:16.535400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b2440-bb9d-4db9-9d67-5e547d05b14d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:16.535804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b2440-bb9d-4db9-9d67-5e547d05b14d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:16.536958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b2440-bb9d-4db9-9d67-5e547d05b14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:16.547733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 836/1687] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:16.548893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34775fad-7448-413f-96dd-5b9fd64c8a78 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:16.557724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-34775fad-7448-413f-96dd-5b9fd64c8a78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:16.610773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-374b2440-bb9d-4db9-9d67-5e547d05b14d None None] MFA Rules not processed for user `bf67549bf7e443c6b77eaf71fb6b709c`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:16.614241 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-374b2440-bb9d-4db9-9d67-5e547d05b14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:16.735193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34775fad-7448-413f-96dd-5b9fd64c8a78 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:16.766166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:16.769234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:16.771246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:16.773186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] PATH_INFO: `/v3/users/2ca986b3bf7f4877badb5828db87dc4c` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:16.776198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca986b3bf7f4877badb5828db87dc4c)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:16.783804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:16.797683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 852/1688] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:16.921408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca986b3bf7f4877badb5828db87dc4c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:16.972914 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca986b3bf7f4877badb5828db87dc4c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:16.974260 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca986b3bf7f4877badb5828db87dc4c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:16.995127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca986b3bf7f4877badb5828db87dc4c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:17.043543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca986b3bf7f4877badb5828db87dc4c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:17.062468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 837/1689] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:16 2022] DELETE /identity/v3/users/2ca986b3bf7f4877badb5828db87dc4c => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:17.063280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d5aba-4c42-4226-a45e-046504dd9f47 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:17.064545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d5aba-4c42-4226-a45e-046504dd9f47 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:17.065102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d5aba-4c42-4226-a45e-046504dd9f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:17.112792 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-706d5aba-4c42-4226-a45e-046504dd9f47 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:17.115560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-706d5aba-4c42-4226-a45e-046504dd9f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:17.272556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:17.274202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:17.307482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 853/1690] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:17.367984 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d5aba-4c42-4226-a45e-046504dd9f47 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:17.369752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-706d5aba-4c42-4226-a45e-046504dd9f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:17.414037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:17.419990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf3a17-4f53-49a0-9d8b-13d93c839d37 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:17.426146 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2a00b-f766-4121-9f6a-1b0b128cc78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:17.427799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2a00b-f766-4121-9f6a-1b0b128cc78b service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:17.428529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2a00b-f766-4121-9f6a-1b0b128cc78b service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:17.429081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2a00b-f766-4121-9f6a-1b0b128cc78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:17.439378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:17.477061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2a00b-f766-4121-9f6a-1b0b128cc78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:17.478942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2a00b-f766-4121-9f6a-1b0b128cc78b service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:17.492423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 838/1691] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:49:17.494016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5438d8a-0cb2-4b8d-83a9-8088721a59d6 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:17.494511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5438d8a-0cb2-4b8d-83a9-8088721a59d6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:17.494950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5438d8a-0cb2-4b8d-83a9-8088721a59d6 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:17.496683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 839/1692] 10.222.0.20 () {54 vars in 960 bytes} [Tue Jul 12 03:49:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:49:17.498552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5438d8a-0cb2-4b8d-83a9-8088721a59d6 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:17.508192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a5438d8a-0cb2-4b8d-83a9-8088721a59d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:17.539662 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d5aba-4c42-4226-a45e-046504dd9f47 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:17.543852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5438d8a-0cb2-4b8d-83a9-8088721a59d6 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:17.548781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:17.549776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:17.550059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:17.550330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] PATH_INFO: `/v3/users/6722ba26f2fc405385b7713cdbc95cdc` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:17.551567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6722ba26f2fc405385b7713cdbc95cdc)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:17.554878 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:17.564068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba2b23-f1f9-4e99-b067-7f820e49c10c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:17.565372 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba2b23-f1f9-4e99-b067-7f820e49c10c service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:17.565867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba2b23-f1f9-4e99-b067-7f820e49c10c service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:17.566363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba2b23-f1f9-4e99-b067-7f820e49c10c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:17.568876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-37ba2b23-f1f9-4e99-b067-7f820e49c10c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:17.719996 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6722ba26f2fc405385b7713cdbc95cdc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:17.742723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba2b23-f1f9-4e99-b067-7f820e49c10c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:17.744798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba2b23-f1f9-4e99-b067-7f820e49c10c service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:17.747078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6722ba26f2fc405385b7713cdbc95cdc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:17.747779 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6722ba26f2fc405385b7713cdbc95cdc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:17.762559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 854/1693] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:49:17.781323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba2b23-f1f9-4e99-b067-7f820e49c10c service nova] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:17.781912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6722ba26f2fc405385b7713cdbc95cdc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:17.787244 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-37ba2b23-f1f9-4e99-b067-7f820e49c10c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:17.814287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6722ba26f2fc405385b7713cdbc95cdc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:17.822945 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 840/1694] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:17 2022] DELETE /identity/v3/users/6722ba26f2fc405385b7713cdbc95cdc => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:17.822945 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:17.825695 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:18.032835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba2b23-f1f9-4e99-b067-7f820e49c10c service nova] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:18.122173 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b004b90-e7c8-4ce7-b717-c667ec9c2df3 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:18.142215 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d72e5d-d3e9-4bd1-adff-3285d6c86d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:18.143159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d72e5d-d3e9-4bd1-adff-3285d6c86d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:18.143495 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d72e5d-d3e9-4bd1-adff-3285d6c86d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:18.146344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d72e5d-d3e9-4bd1-adff-3285d6c86d1a admin admin] PATH_INFO: `/v3/users/ea57984dbdb442cb92b801d48586f176` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:18.149862 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d72e5d-d3e9-4bd1-adff-3285d6c86d1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea57984dbdb442cb92b801d48586f176)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:18.155604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d72e5d-d3e9-4bd1-adff-3285d6c86d1a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:18.185590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:18.193585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea681edb-5391-4009-8571-109a3954aa2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:18.195329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea681edb-5391-4009-8571-109a3954aa2b service placement] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:18.195523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea681edb-5391-4009-8571-109a3954aa2b service placement] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:18.195768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea681edb-5391-4009-8571-109a3954aa2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:18.200605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ea681edb-5391-4009-8571-109a3954aa2b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:18.212684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea681edb-5391-4009-8571-109a3954aa2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:18.214373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea681edb-5391-4009-8571-109a3954aa2b service placement] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:18.228707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 855/1695] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:18.230102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca771172-028f-4a46-b733-57c79f3e2bbb None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:18.230394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca771172-028f-4a46-b733-57c79f3e2bbb None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:18.230670 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca771172-028f-4a46-b733-57c79f3e2bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:18.251346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-12d72e5d-d3e9-4bd1-adff-3285d6c86d1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea57984dbdb442cb92b801d48586f176'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:18.265594 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-12d72e5d-d3e9-4bd1-adff-3285d6c86d1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea57984dbdb442cb92b801d48586f176'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:18.266467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-12d72e5d-d3e9-4bd1-adff-3285d6c86d1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea57984dbdb442cb92b801d48586f176'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:18.267430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-ca771172-028f-4a46-b733-57c79f3e2bbb None None] MFA Rules not processed for user `eded06390a2744c9aa52f77e3bfc2db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:18.268875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ca771172-028f-4a46-b733-57c79f3e2bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:18.280991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-12d72e5d-d3e9-4bd1-adff-3285d6c86d1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea57984dbdb442cb92b801d48586f176'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:18.284742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:18.292880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 856/1696] 10.222.0.20 () {58 vars in 1057 bytes} [Tue Jul 12 03:49:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:18.293849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbb4cf-ab54-452b-a814-9c4fcd3b4668 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:18.294138 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbb4cf-ab54-452b-a814-9c4fcd3b4668 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:18.294475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbb4cf-ab54-452b-a814-9c4fcd3b4668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:18.304762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-12d72e5d-d3e9-4bd1-adff-3285d6c86d1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea57984dbdb442cb92b801d48586f176'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:18.324532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a5dbb4cf-ab54-452b-a814-9c4fcd3b4668 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:18.324532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a5dbb4cf-ab54-452b-a814-9c4fcd3b4668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:18.328990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 841/1697] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:17 2022] DELETE /identity/v3/users/ea57984dbdb442cb92b801d48586f176 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:18.333375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be840a2-dd53-443a-913c-317ae329f9d9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:18.334678 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be840a2-dd53-443a-913c-317ae329f9d9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:18.335417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be840a2-dd53-443a-913c-317ae329f9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:18.408795 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2be840a2-dd53-443a-913c-317ae329f9d9 None None] MFA Rules not processed for user `4ac2825db0c649cab6dcc2e27fbaf80d`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:18.412586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 857/1698] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:18.413687 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbb4cf-ab54-452b-a814-9c4fcd3b4668 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:18.418662 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a5dbb4cf-ab54-452b-a814-9c4fcd3b4668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:18.428467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2be840a2-dd53-443a-913c-317ae329f9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:18.434109 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbb4cf-ab54-452b-a814-9c4fcd3b4668 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:18.441400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:18.442495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:18.442770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:18.443049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] PATH_INFO: `/v3/projects/a9b3d9a1b53b429daedcb93600ef183e` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:18.444451 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9b3d9a1b53b429daedcb93600ef183e)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:18.447373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:18.540673 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9b3d9a1b53b429daedcb93600ef183e', 'request_id': 'req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:18.556042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9b3d9a1b53b429daedcb93600ef183e', 'request_id': 'req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:18.564869 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9b3d9a1b53b429daedcb93600ef183e', 'request_id': 'req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:18.579876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] The token cache is being invalidate because project a9b3d9a1b53b429daedcb93600ef183e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:18.581048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:18.581807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:18.589149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 858/1699] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:18 2022] DELETE /identity/v3/projects/a9b3d9a1b53b429daedcb93600ef183e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:18.593890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:18.598369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:18.658442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 842/1700] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:18.658442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803e694-e2df-4ce5-8f90-f50c95228857 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:18.658840 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803e694-e2df-4ce5-8f90-f50c95228857 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:18.659083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803e694-e2df-4ce5-8f90-f50c95228857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:18.707307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3803e694-e2df-4ce5-8f90-f50c95228857 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:18.710146 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3803e694-e2df-4ce5-8f90-f50c95228857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:18.781914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7bea9-8119-4dc8-8aa1-2c0b3091c4e1 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:18.814050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e47581-dc5c-44d2-8403-07be10d0981d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:18.814898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e47581-dc5c-44d2-8403-07be10d0981d service placement] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:18.815323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e47581-dc5c-44d2-8403-07be10d0981d service placement] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:18.815767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e47581-dc5c-44d2-8403-07be10d0981d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:18.818804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-22e47581-dc5c-44d2-8403-07be10d0981d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:18.859778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e47581-dc5c-44d2-8403-07be10d0981d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:18.861706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e47581-dc5c-44d2-8403-07be10d0981d service placement] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:18.869132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:18.890805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 859/1701] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:18.891730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e47581-dc5c-44d2-8403-07be10d0981d service placement] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:18.894598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-22e47581-dc5c-44d2-8403-07be10d0981d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:18.898762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 843/1702] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:18.902865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803e694-e2df-4ce5-8f90-f50c95228857 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:18.904764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3803e694-e2df-4ce5-8f90-f50c95228857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:18.953101 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803e694-e2df-4ce5-8f90-f50c95228857 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:18.976495 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:18.978169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:18.981078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:18.983225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] PATH_INFO: `/v3/users/deab1fc9cdfb40d7928ac2ea0e6c6003` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:18.989128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deab1fc9cdfb40d7928ac2ea0e6c6003)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:19.013709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:19.123330 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e47581-dc5c-44d2-8403-07be10d0981d service placement] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:19.152439 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb02e2-1553-444d-b024-08b46d62309d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:19.165009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb02e2-1553-444d-b024-08b46d62309d service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:19.165874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb02e2-1553-444d-b024-08b46d62309d service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:19.166164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb02e2-1553-444d-b024-08b46d62309d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:19.171062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d4cb02e2-1553-444d-b024-08b46d62309d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:19.178536 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deab1fc9cdfb40d7928ac2ea0e6c6003'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:19.218106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deab1fc9cdfb40d7928ac2ea0e6c6003'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:19.219257 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deab1fc9cdfb40d7928ac2ea0e6c6003'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:19.242028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb02e2-1553-444d-b024-08b46d62309d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:19.245555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deab1fc9cdfb40d7928ac2ea0e6c6003'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:19.247557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb02e2-1553-444d-b024-08b46d62309d service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:19.275493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deab1fc9cdfb40d7928ac2ea0e6c6003'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:19.287076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:19.296745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 844/1703] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:18 2022] DELETE /identity/v3/users/deab1fc9cdfb40d7928ac2ea0e6c6003 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:19.299465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:19.305846 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:19.307996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 860/1704] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:19.499471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a902708-245b-4bec-8b26-ef6e6f9eda37 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:19.505478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051adc5-de0c-4025-b4b1-e74035e0254a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:19.506548 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051adc5-de0c-4025-b4b1-e74035e0254a service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:19.511153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051adc5-de0c-4025-b4b1-e74035e0254a service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:19.511904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051adc5-de0c-4025-b4b1-e74035e0254a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:19.518703 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e051adc5-de0c-4025-b4b1-e74035e0254a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:19.721019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051adc5-de0c-4025-b4b1-e74035e0254a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:19.727117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051adc5-de0c-4025-b4b1-e74035e0254a service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:19.757066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 845/1705] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:20.719413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb02e2-1553-444d-b024-08b46d62309d service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:20.720926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d4cb02e2-1553-444d-b024-08b46d62309d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:20.754230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb02e2-1553-444d-b024-08b46d62309d service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:20.776889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f3e1a-ef66-4266-8e87-878ffee8ee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:20.777801 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f3e1a-ef66-4266-8e87-878ffee8ee61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:20.779619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f3e1a-ef66-4266-8e87-878ffee8ee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:20.780079 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f3e1a-ef66-4266-8e87-878ffee8ee61 admin admin] PATH_INFO: `/v3/projects/60bc2d22a5ee48d09fa2ef945593079c` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:20.789175 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f3e1a-ef66-4266-8e87-878ffee8ee61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60bc2d22a5ee48d09fa2ef945593079c)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:20.791116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f3e1a-ef66-4266-8e87-878ffee8ee61 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:20.920947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f46f3e1a-ef66-4266-8e87-878ffee8ee61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60bc2d22a5ee48d09fa2ef945593079c', 'request_id': 'req-f46f3e1a-ef66-4266-8e87-878ffee8ee61'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:20.936540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f46f3e1a-ef66-4266-8e87-878ffee8ee61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60bc2d22a5ee48d09fa2ef945593079c', 'request_id': 'req-f46f3e1a-ef66-4266-8e87-878ffee8ee61'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:20.975004 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051adc5-de0c-4025-b4b1-e74035e0254a service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:20.978005 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e051adc5-de0c-4025-b4b1-e74035e0254a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:20.978279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f46f3e1a-ef66-4266-8e87-878ffee8ee61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60bc2d22a5ee48d09fa2ef945593079c', 'request_id': 'req-f46f3e1a-ef66-4266-8e87-878ffee8ee61'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.000794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051adc5-de0c-4025-b4b1-e74035e0254a service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:21.005367 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f46f3e1a-ef66-4266-8e87-878ffee8ee61 admin admin] The token cache is being invalidate because project 60bc2d22a5ee48d09fa2ef945593079c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:21.008453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f46f3e1a-ef66-4266-8e87-878ffee8ee61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.010656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f46f3e1a-ef66-4266-8e87-878ffee8ee61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.015149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 861/1706] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:20 2022] DELETE /identity/v3/projects/60bc2d22a5ee48d09fa2ef945593079c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:21.020560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831afa0-1db8-4db3-aa6a-b36c0911ab2f None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:21.021157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831afa0-1db8-4db3-aa6a-b36c0911ab2f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:21.021648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831afa0-1db8-4db3-aa6a-b36c0911ab2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:21.070487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d831afa0-1db8-4db3-aa6a-b36c0911ab2f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:21.071994 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d831afa0-1db8-4db3-aa6a-b36c0911ab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:21.167687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:21.169025 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:21.169468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:21.169883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] PATH_INFO: `/v3/users/249f1cf3635a45ec8181a2a7e2ef1c1c` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:21.175971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] RBAC: Authorizing `identity:delete_user(user_id=249f1cf3635a45ec8181a2a7e2ef1c1c)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:21.178693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:21.215503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 862/1707] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:21.224518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7b42e-7f4d-4995-8695-910fe85260e0 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:21.225325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7b42e-7f4d-4995-8695-910fe85260e0 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:21.225902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7b42e-7f4d-4995-8695-910fe85260e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:21.274473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-99a7b42e-7f4d-4995-8695-910fe85260e0 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:21.276413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-99a7b42e-7f4d-4995-8695-910fe85260e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:21.289592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 863/1708] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:21.293441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '249f1cf3635a45ec8181a2a7e2ef1c1c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.295192 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7b42e-7f4d-4995-8695-910fe85260e0 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:21.299269 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:21.310983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '249f1cf3635a45ec8181a2a7e2ef1c1c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.311589 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '249f1cf3635a45ec8181a2a7e2ef1c1c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.324413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '249f1cf3635a45ec8181a2a7e2ef1c1c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.327477 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7b42e-7f4d-4995-8695-910fe85260e0 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:21.335070 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b468df-7d95-4434-8284-0a11f6fed7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:21.336123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b468df-7d95-4434-8284-0a11f6fed7ba admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:21.336576 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b468df-7d95-4434-8284-0a11f6fed7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:21.337034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b468df-7d95-4434-8284-0a11f6fed7ba admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:21.337968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b468df-7d95-4434-8284-0a11f6fed7ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:21.338351 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b468df-7d95-4434-8284-0a11f6fed7ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:21.340647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b468df-7d95-4434-8284-0a11f6fed7ba admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:21.358099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 864/1709] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:21.359240 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b468df-7d95-4434-8284-0a11f6fed7ba admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:21.361596 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-87b468df-7d95-4434-8284-0a11f6fed7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:21.368913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '249f1cf3635a45ec8181a2a7e2ef1c1c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.376562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:21.381951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b468df-7d95-4434-8284-0a11f6fed7ba admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:21.382752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 846/1710] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:20 2022] DELETE /identity/v3/users/249f1cf3635a45ec8181a2a7e2ef1c1c => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:21.383677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:21.388542 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:21.541282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12122011-7bba-4005-83fa-dcf19647617d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:21.552838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db543223-fa0e-4fd7-8ea7-7a38fce9a6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:21.553687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db543223-fa0e-4fd7-8ea7-7a38fce9a6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:21.554067 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db543223-fa0e-4fd7-8ea7-7a38fce9a6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:21.554346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db543223-fa0e-4fd7-8ea7-7a38fce9a6e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:21.555094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db543223-fa0e-4fd7-8ea7-7a38fce9a6e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:21.557349 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db543223-fa0e-4fd7-8ea7-7a38fce9a6e3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:21.558917 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:21.565144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:21.565841 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:21.566938 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] PATH_INFO: `/v3/projects/dee515d5612e4524963a0f2c38791419` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:21.573761 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dee515d5612e4524963a0f2c38791419)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:21.581871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:21.656135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 847/1711] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:21 2022] POST /identity/v3/projects => generated 379 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:21.662439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22caac66-406b-45ce-863b-e59931720399 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:21.663870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22caac66-406b-45ce-863b-e59931720399 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:21.666482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22caac66-406b-45ce-863b-e59931720399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:21.720030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-22caac66-406b-45ce-863b-e59931720399 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:21.730625 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-22caac66-406b-45ce-863b-e59931720399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:21.849207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dee515d5612e4524963a0f2c38791419', 'request_id': 'req-14461caa-8308-4eb2-9e94-69e022f4bb47'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.871396 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dee515d5612e4524963a0f2c38791419', 'request_id': 'req-14461caa-8308-4eb2-9e94-69e022f4bb47'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.881890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 848/1712] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:21.883170 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22caac66-406b-45ce-863b-e59931720399 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:21.884793 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-22caac66-406b-45ce-863b-e59931720399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:21.891291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dee515d5612e4524963a0f2c38791419', 'request_id': 'req-14461caa-8308-4eb2-9e94-69e022f4bb47'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.920046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22caac66-406b-45ce-863b-e59931720399 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:21.924077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] The token cache is being invalidate because project dee515d5612e4524963a0f2c38791419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:21.924077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.924906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:21.927789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 865/1713] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:21 2022] DELETE /identity/v3/projects/dee515d5612e4524963a0f2c38791419 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:21.931242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:21.936142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:22.058664 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:22.067383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:22.068607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:22.070567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:22.071131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] PATH_INFO: `/v3/projects/81be5c4d248246a28ca48d7448a4ca9f` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:22.075240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=81be5c4d248246a28ca48d7448a4ca9f)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:22.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: WARNING py.warnings [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:49:22.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: warnings.warn(msg) Jul 12 03:49:22.076724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: Jul 12 03:49:22.078575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:22.173278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14461caa-8308-4eb2-9e94-69e022f4bb47 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:22.184945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901bb624-2fd1-4676-ba11-031502341e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:22.187011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901bb624-2fd1-4676-ba11-031502341e77 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:22.187406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901bb624-2fd1-4676-ba11-031502341e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:22.188691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901bb624-2fd1-4676-ba11-031502341e77 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:22.189800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901bb624-2fd1-4676-ba11-031502341e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:22.194908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901bb624-2fd1-4676-ba11-031502341e77 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:22.246144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 866/1714] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:21 2022] POST /identity/v3/users => generated 290 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:22.247120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901bb624-2fd1-4676-ba11-031502341e77 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:22.249078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-901bb624-2fd1-4676-ba11-031502341e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:22.277534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81be5c4d248246a28ca48d7448a4ca9f', 'request_id': 'req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:22.305413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81be5c4d248246a28ca48d7448a4ca9f', 'request_id': 'req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:22.321968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81be5c4d248246a28ca48d7448a4ca9f', 'request_id': 'req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:22.348772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:22.385347 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] The token cache is being invalidate because project 81be5c4d248246a28ca48d7448a4ca9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:22.386349 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:22.388729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9cc92b5c-1a75-4b7e-ac22-5ab8c7877c5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:22.397582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 849/1715] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:21 2022] DELETE /identity/v3/projects/81be5c4d248246a28ca48d7448a4ca9f => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:22.399110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e979b6-0ea6-4cd7-a47a-c2960c252d24 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:22.399558 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e979b6-0ea6-4cd7-a47a-c2960c252d24 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:22.401024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e979b6-0ea6-4cd7-a47a-c2960c252d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:22.437197 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b2e979b6-0ea6-4cd7-a47a-c2960c252d24 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:22.438609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b2e979b6-0ea6-4cd7-a47a-c2960c252d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:22.464993 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901bb624-2fd1-4676-ba11-031502341e77 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:22.603063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865ec9d-4717-4cd2-8ef0-29786a064a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:22.603753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865ec9d-4717-4cd2-8ef0-29786a064a35 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:22.604061 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865ec9d-4717-4cd2-8ef0-29786a064a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:22.604345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865ec9d-4717-4cd2-8ef0-29786a064a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:22.609168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e865ec9d-4717-4cd2-8ef0-29786a064a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:22.680325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 850/1716] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:22.681282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e979b6-0ea6-4cd7-a47a-c2960c252d24 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:22.682794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b2e979b6-0ea6-4cd7-a47a-c2960c252d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:22.704093 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e979b6-0ea6-4cd7-a47a-c2960c252d24 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:22.713357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f0585-18db-4f1c-a214-5697eb219ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:22.714739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f0585-18db-4f1c-a214-5697eb219ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:22.716972 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f0585-18db-4f1c-a214-5697eb219ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:22.717284 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f0585-18db-4f1c-a214-5697eb219ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:22.720353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f0585-18db-4f1c-a214-5697eb219ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:22.722762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f0585-18db-4f1c-a214-5697eb219ac9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:22.747689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 851/1717] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:22.747689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f0585-18db-4f1c-a214-5697eb219ac9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:22.748115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865ec9d-4717-4cd2-8ef0-29786a064a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:22.750452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-363f0585-18db-4f1c-a214-5697eb219ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:22.752200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865ec9d-4717-4cd2-8ef0-29786a064a35 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:22.765849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 867/1718] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:22.770115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73047ab8-e4ed-4cab-990c-04372c259482 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:22.770422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73047ab8-e4ed-4cab-990c-04372c259482 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:22.771195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73047ab8-e4ed-4cab-990c-04372c259482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:22.778415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f0585-18db-4f1c-a214-5697eb219ac9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:22.798534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-73047ab8-e4ed-4cab-990c-04372c259482 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:22.800162 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-73047ab8-e4ed-4cab-990c-04372c259482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:22.807672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7566dc-ef16-4306-b311-99626ae035dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:22.808729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7566dc-ef16-4306-b311-99626ae035dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:22.809014 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7566dc-ef16-4306-b311-99626ae035dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:22.809291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7566dc-ef16-4306-b311-99626ae035dd admin admin] PATH_INFO: `/v3/users/24423b12130f416ab15d32c4f2e854d1` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:22.811382 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7566dc-ef16-4306-b311-99626ae035dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=24423b12130f416ab15d32c4f2e854d1)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:22.813496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7566dc-ef16-4306-b311-99626ae035dd admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:22.816865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 868/1719] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:22.818054 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73047ab8-e4ed-4cab-990c-04372c259482 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:22.833209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73047ab8-e4ed-4cab-990c-04372c259482 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:22.840314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c26104-f819-4994-9de2-4daf27861fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:22.841018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c26104-f819-4994-9de2-4daf27861fcb admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:22.841591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c26104-f819-4994-9de2-4daf27861fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:22.841871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c26104-f819-4994-9de2-4daf27861fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:22.843248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c26104-f819-4994-9de2-4daf27861fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:22.844975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c26104-f819-4994-9de2-4daf27861fcb admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:22.864611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 869/1720] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:22.864611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c26104-f819-4994-9de2-4daf27861fcb admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:22.875611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c26104-f819-4994-9de2-4daf27861fcb admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:22.889233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda50ad2-a0d2-44c6-aa9d-0e843789f38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:22.889945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda50ad2-a0d2-44c6-aa9d-0e843789f38f admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:22.892268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda50ad2-a0d2-44c6-aa9d-0e843789f38f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:22.893897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda50ad2-a0d2-44c6-aa9d-0e843789f38f admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:22.894671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda50ad2-a0d2-44c6-aa9d-0e843789f38f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:22.894985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda50ad2-a0d2-44c6-aa9d-0e843789f38f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:22.897472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda50ad2-a0d2-44c6-aa9d-0e843789f38f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:22.928141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 870/1721] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:22.930605 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8d7566dc-ef16-4306-b311-99626ae035dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24423b12130f416ab15d32c4f2e854d1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:22.932591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda50ad2-a0d2-44c6-aa9d-0e843789f38f admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:22.949122 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda50ad2-a0d2-44c6-aa9d-0e843789f38f admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:22.956028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0f752-1b91-44ad-910a-e301da14fe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:22.958494 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8d7566dc-ef16-4306-b311-99626ae035dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24423b12130f416ab15d32c4f2e854d1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:22.959364 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8d7566dc-ef16-4306-b311-99626ae035dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24423b12130f416ab15d32c4f2e854d1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:22.963844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0f752-1b91-44ad-910a-e301da14fe6a admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:22.963844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0f752-1b91-44ad-910a-e301da14fe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:22.963844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0f752-1b91-44ad-910a-e301da14fe6a admin admin] PATH_INFO: `/v3/projects/28fae6efdcd447baa6a9df2c36bb976d/users/35655269c1d3479080d10ceec817bbf9/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:22.973002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8d7566dc-ef16-4306-b311-99626ae035dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24423b12130f416ab15d32c4f2e854d1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:22.986587 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:23.002493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0f752-1b91-44ad-910a-e301da14fe6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=28fae6efdcd447baa6a9df2c36bb976d, user_id=35655269c1d3479080d10ceec817bbf9, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:23.004214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8d7566dc-ef16-4306-b311-99626ae035dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24423b12130f416ab15d32c4f2e854d1'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:23.014336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0f752-1b91-44ad-910a-e301da14fe6a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:23.024232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 852/1722] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:22 2022] DELETE /identity/v3/users/24423b12130f416ab15d32c4f2e854d1 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:23.026265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927f068-325b-4b91-8449-c07dec53384d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:23.026677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927f068-325b-4b91-8449-c07dec53384d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:23.027522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927f068-325b-4b91-8449-c07dec53384d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:23.049415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 871/1723] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:22 2022] PUT /identity/v3/projects/28fae6efdcd447baa6a9df2c36bb976d/users/35655269c1d3479080d10ceec817bbf9/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:23.049415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0f752-1b91-44ad-910a-e301da14fe6a admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:23.054999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6be0f752-1b91-44ad-910a-e301da14fe6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:23.146194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:23.199030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3927f068-325b-4b91-8449-c07dec53384d None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:23.201704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3927f068-325b-4b91-8449-c07dec53384d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:23.219158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0f752-1b91-44ad-910a-e301da14fe6a admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:23.235501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99b83b-16e4-4439-b8d1-3ccabc4294ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:23.236265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99b83b-16e4-4439-b8d1-3ccabc4294ee admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:23.236516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99b83b-16e4-4439-b8d1-3ccabc4294ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:23.236845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99b83b-16e4-4439-b8d1-3ccabc4294ee admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:23.238274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99b83b-16e4-4439-b8d1-3ccabc4294ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:23.240671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99b83b-16e4-4439-b8d1-3ccabc4294ee admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:23.302360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 872/1724] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:23 2022] POST /identity/v3/projects => generated 367 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:23.305712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d10cc-185d-4b2b-bb95-f5ddef9b5d35 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:23.308638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d10cc-185d-4b2b-bb95-f5ddef9b5d35 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:23.311530 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d10cc-185d-4b2b-bb95-f5ddef9b5d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:23.354108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-004d10cc-185d-4b2b-bb95-f5ddef9b5d35 None None] MFA Rules not processed for user `35655269c1d3479080d10ceec817bbf9`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:23.359251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-004d10cc-185d-4b2b-bb95-f5ddef9b5d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:23.374090 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 853/1725] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:49:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:23.376193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927f068-325b-4b91-8449-c07dec53384d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:23.405876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927f068-325b-4b91-8449-c07dec53384d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:23.495963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 873/1726] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:23.499176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d10cc-185d-4b2b-bb95-f5ddef9b5d35 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:23.505582 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-004d10cc-185d-4b2b-bb95-f5ddef9b5d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:23.554130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ac565-e95e-4d6d-8efb-b59764eaed6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:23.554959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ac565-e95e-4d6d-8efb-b59764eaed6f admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:23.555620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ac565-e95e-4d6d-8efb-b59764eaed6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:23.555620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ac565-e95e-4d6d-8efb-b59764eaed6f admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:23.558471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ac565-e95e-4d6d-8efb-b59764eaed6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:23.562331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ac565-e95e-4d6d-8efb-b59764eaed6f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:23.622311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 854/1727] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:23 2022] POST /identity/v3/users => generated 284 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:23.631921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ac565-e95e-4d6d-8efb-b59764eaed6f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:23.676323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d10cc-185d-4b2b-bb95-f5ddef9b5d35 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:23.682040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ac565-e95e-4d6d-8efb-b59764eaed6f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:23.686166 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce49bf-b3fd-4b93-a120-d2b432c61ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:23.687110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce49bf-b3fd-4b93-a120-d2b432c61ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:23.687654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce49bf-b3fd-4b93-a120-d2b432c61ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:23.688120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce49bf-b3fd-4b93-a120-d2b432c61ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:23.688925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce49bf-b3fd-4b93-a120-d2b432c61ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:23.691282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce49bf-b3fd-4b93-a120-d2b432c61ea7 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:23.708297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9469b-15e8-4a48-8d54-9e536e170b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:23.715303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9469b-15e8-4a48-8d54-9e536e170b0b admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:23.716933 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9469b-15e8-4a48-8d54-9e536e170b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:23.718025 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9469b-15e8-4a48-8d54-9e536e170b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:23.733246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9469b-15e8-4a48-8d54-9e536e170b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:23.735142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9469b-15e8-4a48-8d54-9e536e170b0b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:23.814894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 855/1728] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:23.816734 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 874/1729] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:23 2022] POST /identity/v3/projects => generated 379 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:23.823413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce49bf-b3fd-4b93-a120-d2b432c61ea7 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:23.830015 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9469b-15e8-4a48-8d54-9e536e170b0b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:23.834905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-28a9469b-15e8-4a48-8d54-9e536e170b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:23.899736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:23.937290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce49bf-b3fd-4b93-a120-d2b432c61ea7 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:23.943784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91e244-9298-4717-8066-4196e209347c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:23.947925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91e244-9298-4717-8066-4196e209347c admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:23.948899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91e244-9298-4717-8066-4196e209347c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:23.949657 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91e244-9298-4717-8066-4196e209347c admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:23.951894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91e244-9298-4717-8066-4196e209347c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:23.956315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91e244-9298-4717-8066-4196e209347c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:24.013352 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 875/1730] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:24.019197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91e244-9298-4717-8066-4196e209347c admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:24.032226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:24.051498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91e244-9298-4717-8066-4196e209347c admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:24.071102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a68148-951a-4e6e-82f4-28cc5d0551e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:24.073674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a68148-951a-4e6e-82f4-28cc5d0551e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:24.073674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a68148-951a-4e6e-82f4-28cc5d0551e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:24.073674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a68148-951a-4e6e-82f4-28cc5d0551e2 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:24.081455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a68148-951a-4e6e-82f4-28cc5d0551e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:24.091370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a68148-951a-4e6e-82f4-28cc5d0551e2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:24.167246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 876/1731] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:24 2022] POST /identity/v3/users => generated 296 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:24.167246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a68148-951a-4e6e-82f4-28cc5d0551e2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:24.172856 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9469b-15e8-4a48-8d54-9e536e170b0b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:24.198800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a68148-951a-4e6e-82f4-28cc5d0551e2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:24.212867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d31f24-e04d-441f-8f8e-86f11e846e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:24.216222 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d31f24-e04d-441f-8f8e-86f11e846e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:24.216565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d31f24-e04d-441f-8f8e-86f11e846e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:24.216900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d31f24-e04d-441f-8f8e-86f11e846e77 admin admin] PATH_INFO: `/v3/projects/93c8aaa9f9a64f7d8cbbd40054419fb2/users/7daefac2eadd4eeea21d2d5a5e2d01db/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:24.262984 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d31f24-e04d-441f-8f8e-86f11e846e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c8aaa9f9a64f7d8cbbd40054419fb2, user_id=7daefac2eadd4eeea21d2d5a5e2d01db, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:24.272941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d31f24-e04d-441f-8f8e-86f11e846e77 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:24.317526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 877/1732] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:24 2022] PUT /identity/v3/projects/93c8aaa9f9a64f7d8cbbd40054419fb2/users/7daefac2eadd4eeea21d2d5a5e2d01db/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:24.323915 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d31f24-e04d-441f-8f8e-86f11e846e77 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:24.378265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649eaf7-d2a9-40ee-ba9b-b87fad2312cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:24.378957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649eaf7-d2a9-40ee-ba9b-b87fad2312cd service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:24.379245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649eaf7-d2a9-40ee-ba9b-b87fad2312cd service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:24.379525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649eaf7-d2a9-40ee-ba9b-b87fad2312cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:24.475086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649eaf7-d2a9-40ee-ba9b-b87fad2312cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:24.477260 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649eaf7-d2a9-40ee-ba9b-b87fad2312cd service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:24.502809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 856/1733] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:49:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:49:24.513774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7b219-c942-46f5-b1e2-54058deb13c2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:24.516047 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7b219-c942-46f5-b1e2-54058deb13c2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:24.516412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7b219-c942-46f5-b1e2-54058deb13c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:24.533053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d31f24-e04d-441f-8f8e-86f11e846e77 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:24.591467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5fe75-a003-48da-b4a9-d9ebf27a4f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:24.593933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5fe75-a003-48da-b4a9-d9ebf27a4f99 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:24.594263 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5fe75-a003-48da-b4a9-d9ebf27a4f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:24.595746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5fe75-a003-48da-b4a9-d9ebf27a4f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:24.601164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5fe75-a003-48da-b4a9-d9ebf27a4f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:24.607262 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5fe75-a003-48da-b4a9-d9ebf27a4f99 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:24.635143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4cf7b219-c942-46f5-b1e2-54058deb13c2 None None] MFA Rules not processed for user `7daefac2eadd4eeea21d2d5a5e2d01db`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:24.642139 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4cf7b219-c942-46f5-b1e2-54058deb13c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:24.671215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 878/1734] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:24.674025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5fe75-a003-48da-b4a9-d9ebf27a4f99 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:24.741039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5fe75-a003-48da-b4a9-d9ebf27a4f99 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:24.769902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08e7b0-87e7-40dc-97ad-d83afe2d8ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:24.770893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08e7b0-87e7-40dc-97ad-d83afe2d8ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:24.771279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08e7b0-87e7-40dc-97ad-d83afe2d8ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:24.778988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08e7b0-87e7-40dc-97ad-d83afe2d8ed0 admin admin] PATH_INFO: `/v3/projects/497bd535a4b04d668f1cec811c136a3d/users/1c871a7e51634971b5e963d3550df171/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:24.826623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:24.853085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08e7b0-87e7-40dc-97ad-d83afe2d8ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=497bd535a4b04d668f1cec811c136a3d, user_id=1c871a7e51634971b5e963d3550df171, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:24.863236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 857/1735] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:24.864616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1df885-cead-43a7-af29-d2fe6ea1b6f7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:24.865561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1df885-cead-43a7-af29-d2fe6ea1b6f7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:24.865913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1df885-cead-43a7-af29-d2fe6ea1b6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:24.866914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08e7b0-87e7-40dc-97ad-d83afe2d8ed0 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:24.910547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 879/1736] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:24 2022] PUT /identity/v3/projects/497bd535a4b04d668f1cec811c136a3d/users/1c871a7e51634971b5e963d3550df171/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:24.918956 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08e7b0-87e7-40dc-97ad-d83afe2d8ed0 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:24.947568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ea1df885-cead-43a7-af29-d2fe6ea1b6f7 None None] MFA Rules not processed for user `139be21e7f644b5796d30158851ebd00`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:24.949866 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ea1df885-cead-43a7-af29-d2fe6ea1b6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:25.085238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 858/1737] 10.222.0.44 () {58 vars in 1015 bytes} [Tue Jul 12 03:49:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:25.086851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1df885-cead-43a7-af29-d2fe6ea1b6f7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:25.127727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08e7b0-87e7-40dc-97ad-d83afe2d8ed0 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:25.139345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e933c0-084e-427e-af50-59ac3d49a3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:25.143790 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e933c0-084e-427e-af50-59ac3d49a3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:25.144115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e933c0-084e-427e-af50-59ac3d49a3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:25.147153 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e933c0-084e-427e-af50-59ac3d49a3e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:25.147910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e933c0-084e-427e-af50-59ac3d49a3e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:25.154525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e933c0-084e-427e-af50-59ac3d49a3e3 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:25.168467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:25.232455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 880/1738] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:24 2022] POST /identity/v3/projects => generated 369 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:25.236622 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e933c0-084e-427e-af50-59ac3d49a3e3 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:25.249087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1df885-cead-43a7-af29-d2fe6ea1b6f7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:25.385461 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e933c0-084e-427e-af50-59ac3d49a3e3 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:25.390855 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915df10-c058-4a31-8bf2-367bada24b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:25.391548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915df10-c058-4a31-8bf2-367bada24b22 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:25.391912 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915df10-c058-4a31-8bf2-367bada24b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:25.392135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166af1ff-c9ce-401f-850c-4478c51e5cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:25.392651 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915df10-c058-4a31-8bf2-367bada24b22 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:25.392968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915df10-c058-4a31-8bf2-367bada24b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:25.393746 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166af1ff-c9ce-401f-850c-4478c51e5cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:25.394883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166af1ff-c9ce-401f-850c-4478c51e5cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:25.395120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915df10-c058-4a31-8bf2-367bada24b22 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:25.397912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166af1ff-c9ce-401f-850c-4478c51e5cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:25.399066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166af1ff-c9ce-401f-850c-4478c51e5cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:25.401015 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166af1ff-c9ce-401f-850c-4478c51e5cb5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:25.424705 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 859/1739] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:25.424705 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166af1ff-c9ce-401f-850c-4478c51e5cb5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:25.432892 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-166af1ff-c9ce-401f-850c-4478c51e5cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:25.437220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 881/1740] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:25 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:25.441648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915df10-c058-4a31-8bf2-367bada24b22 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:25.475989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915df10-c058-4a31-8bf2-367bada24b22 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:25.489645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34877180-475c-4311-8026-b57fdd897580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:25.493307 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34877180-475c-4311-8026-b57fdd897580 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:25.493807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34877180-475c-4311-8026-b57fdd897580 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:25.495693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34877180-475c-4311-8026-b57fdd897580 admin admin] PATH_INFO: `/v3/projects/497bd535a4b04d668f1cec811c136a3d/users/1c871a7e51634971b5e963d3550df171/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:25.501044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34877180-475c-4311-8026-b57fdd897580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=497bd535a4b04d668f1cec811c136a3d, user_id=1c871a7e51634971b5e963d3550df171, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:25.509703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34877180-475c-4311-8026-b57fdd897580 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:25.556238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:25.582364 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166af1ff-c9ce-401f-850c-4478c51e5cb5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:25.590366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 882/1741] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:25 2022] PUT /identity/v3/projects/497bd535a4b04d668f1cec811c136a3d/users/1c871a7e51634971b5e963d3550df171/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:25.594552 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34877180-475c-4311-8026-b57fdd897580 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:25.769241 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34877180-475c-4311-8026-b57fdd897580 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:25.775977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9cf5c5-3d7f-4fb4-813d-e94e50dce444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:25.776829 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cf5c5-3d7f-4fb4-813d-e94e50dce444 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:25.777206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cf5c5-3d7f-4fb4-813d-e94e50dce444 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:25.777545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cf5c5-3d7f-4fb4-813d-e94e50dce444 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:25.778425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9cf5c5-3d7f-4fb4-813d-e94e50dce444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:25.780815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9cf5c5-3d7f-4fb4-813d-e94e50dce444 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:25.795148 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 883/1742] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:25.796856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4bca5-ee4e-419d-b280-749c0a3dc747 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:25.797282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4bca5-ee4e-419d-b280-749c0a3dc747 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:25.797660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4bca5-ee4e-419d-b280-749c0a3dc747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:25.805678 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ce427-e128-47bc-91af-ae805363ec88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:25.807630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ce427-e128-47bc-91af-ae805363ec88 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:25.809748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ce427-e128-47bc-91af-ae805363ec88 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:25.810251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ce427-e128-47bc-91af-ae805363ec88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:25.834366 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ce427-e128-47bc-91af-ae805363ec88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:25.835554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-e8e4bca5-ee4e-419d-b280-749c0a3dc747 None None] MFA Rules not processed for user `1c871a7e51634971b5e963d3550df171`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:25.837314 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ce427-e128-47bc-91af-ae805363ec88 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:25.837730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e8e4bca5-ee4e-419d-b280-749c0a3dc747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:25.868895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 860/1743] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:25.869657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ce427-e128-47bc-91af-ae805363ec88 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:25.872039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-057ce427-e128-47bc-91af-ae805363ec88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:25.896047 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ce427-e128-47bc-91af-ae805363ec88 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:25.902962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:25.903932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:25.904908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:25.905260 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] PATH_INFO: `/v3/projects/e61b7267acd6432782785446abcb706a` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:25.906479 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e61b7267acd6432782785446abcb706a)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:25.909289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:25.982972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:26.018788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e61b7267acd6432782785446abcb706a', 'request_id': 'req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:26.030660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 884/1744] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:26.031586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4bca5-ee4e-419d-b280-749c0a3dc747 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:26.035129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e61b7267acd6432782785446abcb706a', 'request_id': 'req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:26.046380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e61b7267acd6432782785446abcb706a', 'request_id': 'req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:26.075812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] The token cache is being invalidate because project e61b7267acd6432782785446abcb706a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:26.076163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:26.076685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4bca5-ee4e-419d-b280-749c0a3dc747 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:26.076859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:26.080711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 861/1745] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:25 2022] DELETE /identity/v3/projects/e61b7267acd6432782785446abcb706a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:26.081683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:26.084740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:26.250177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b546cdf7-93e4-48e3-b760-6678f2463ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:26.254322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546cdf7-93e4-48e3-b760-6678f2463ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:26.254709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546cdf7-93e4-48e3-b760-6678f2463ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:26.255272 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b546cdf7-93e4-48e3-b760-6678f2463ef1 admin admin] PATH_INFO: `/v3/projects/2a8a9a1c957746c58fdc77099580a090/users/3151bfdb62ae4824b5bfa1de006ac4de/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:26.282154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b546cdf7-93e4-48e3-b760-6678f2463ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8a9a1c957746c58fdc77099580a090, user_id=3151bfdb62ae4824b5bfa1de006ac4de, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:26.287285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b546cdf7-93e4-48e3-b760-6678f2463ef1 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:26.326492 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6deefe-fc86-47f7-a1c9-378fd4f74c8b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:26.328978 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 885/1746] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:26 2022] PUT /identity/v3/projects/2a8a9a1c957746c58fdc77099580a090/users/3151bfdb62ae4824b5bfa1de006ac4de/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:26.333246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e844e1-7397-4992-8674-0f41b8113efa None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:26.334358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e844e1-7397-4992-8674-0f41b8113efa None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:26.334721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e844e1-7397-4992-8674-0f41b8113efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:26.387339 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-b5e844e1-7397-4992-8674-0f41b8113efa None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:26.390313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b5e844e1-7397-4992-8674-0f41b8113efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:26.442413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:26.470667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d937c3-84f0-4c67-99d5-19216f59f814 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:26.471522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d937c3-84f0-4c67-99d5-19216f59f814 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:26.471825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d937c3-84f0-4c67-99d5-19216f59f814 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:26.472151 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d937c3-84f0-4c67-99d5-19216f59f814 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:26.478847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-28d937c3-84f0-4c67-99d5-19216f59f814 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:26.531456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 886/1747] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:26.532586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e844e1-7397-4992-8674-0f41b8113efa None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:26.534577 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b5e844e1-7397-4992-8674-0f41b8113efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:26.555339 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e844e1-7397-4992-8674-0f41b8113efa None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:26.573709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e426bd7-93ae-41a1-9da0-a37cd0ab3b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:26.574632 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e426bd7-93ae-41a1-9da0-a37cd0ab3b37 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:26.577285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e426bd7-93ae-41a1-9da0-a37cd0ab3b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:26.578146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e426bd7-93ae-41a1-9da0-a37cd0ab3b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:26.582732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e426bd7-93ae-41a1-9da0-a37cd0ab3b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:26.590183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e426bd7-93ae-41a1-9da0-a37cd0ab3b37 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:26.624896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 887/1748] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:26.630487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ecd95-7b9b-42c9-b856-8513b531d7c3 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:26.632079 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ecd95-7b9b-42c9-b856-8513b531d7c3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:26.632506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ecd95-7b9b-42c9-b856-8513b531d7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:26.672239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d937c3-84f0-4c67-99d5-19216f59f814 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:26.675090 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d937c3-84f0-4c67-99d5-19216f59f814 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:26.680699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-742ecd95-7b9b-42c9-b856-8513b531d7c3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:26.681688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-742ecd95-7b9b-42c9-b856-8513b531d7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:26.705759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 888/1749] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:26.710374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ecd95-7b9b-42c9-b856-8513b531d7c3 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:26.711395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 862/1750] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:26.712402 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d937c3-84f0-4c67-99d5-19216f59f814 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:26.747741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d937c3-84f0-4c67-99d5-19216f59f814 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:26.750360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ecd95-7b9b-42c9-b856-8513b531d7c3 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:26.758114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7999e-db81-4c38-98f5-eef2aa69ba44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:26.759579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7999e-db81-4c38-98f5-eef2aa69ba44 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:26.760094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7999e-db81-4c38-98f5-eef2aa69ba44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:26.760581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7999e-db81-4c38-98f5-eef2aa69ba44 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:26.761504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7999e-db81-4c38-98f5-eef2aa69ba44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:26.762952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7999e-db81-4c38-98f5-eef2aa69ba44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:26.764557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbd5d9-44c8-43aa-9a66-a1c6d254b9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:26.767245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbd5d9-44c8-43aa-9a66-a1c6d254b9ba admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:26.768080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbd5d9-44c8-43aa-9a66-a1c6d254b9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:26.768371 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbd5d9-44c8-43aa-9a66-a1c6d254b9ba admin admin] PATH_INFO: `/v3/projects/2a8a9a1c957746c58fdc77099580a090/users/3151bfdb62ae4824b5bfa1de006ac4de/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:26.776307 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbd5d9-44c8-43aa-9a66-a1c6d254b9ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8a9a1c957746c58fdc77099580a090, user_id=3151bfdb62ae4824b5bfa1de006ac4de, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:26.779487 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7999e-db81-4c38-98f5-eef2aa69ba44 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:26.787497 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbd5d9-44c8-43aa-9a66-a1c6d254b9ba admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:26.826359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 863/1751] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:26.829246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 889/1752] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:26 2022] PUT /identity/v3/projects/2a8a9a1c957746c58fdc77099580a090/users/3151bfdb62ae4824b5bfa1de006ac4de/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:26.831225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7999e-db81-4c38-98f5-eef2aa69ba44 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:26.840374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c5324-d456-4d2c-a1b6-12ab52a40dff None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:26.840663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c5324-d456-4d2c-a1b6-12ab52a40dff None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:26.841279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c5324-d456-4d2c-a1b6-12ab52a40dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:26.869865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7999e-db81-4c38-98f5-eef2aa69ba44 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:26.982368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f29c5324-d456-4d2c-a1b6-12ab52a40dff None None] MFA Rules not processed for user `3151bfdb62ae4824b5bfa1de006ac4de`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:26.983787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f29c5324-d456-4d2c-a1b6-12ab52a40dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:27.072876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbf147-aa6e-4b6e-8b72-7efa7fe8c575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:27.073706 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbf147-aa6e-4b6e-8b72-7efa7fe8c575 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:27.074050 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbf147-aa6e-4b6e-8b72-7efa7fe8c575 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:27.074405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbf147-aa6e-4b6e-8b72-7efa7fe8c575 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:27.075306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbf147-aa6e-4b6e-8b72-7efa7fe8c575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:27.077634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbf147-aa6e-4b6e-8b72-7efa7fe8c575 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:27.088036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:27.163707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 864/1753] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:26 2022] POST /identity/v3/projects => generated 363 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:27.165714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbf147-aa6e-4b6e-8b72-7efa7fe8c575 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:27.223799 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 890/1754] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:26 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:27.224783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c5324-d456-4d2c-a1b6-12ab52a40dff None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:27.252448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c5324-d456-4d2c-a1b6-12ab52a40dff None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:27.318643 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbf147-aa6e-4b6e-8b72-7efa7fe8c575 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:27.322458 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6d182-b734-42a8-8eb0-30b26c182e88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:27.323368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6d182-b734-42a8-8eb0-30b26c182e88 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:27.323725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6d182-b734-42a8-8eb0-30b26c182e88 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:27.324164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6d182-b734-42a8-8eb0-30b26c182e88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:27.326353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-91b6d182-b734-42a8-8eb0-30b26c182e88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:27.346643 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:27.409133 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6d182-b734-42a8-8eb0-30b26c182e88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:27.410984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6d182-b734-42a8-8eb0-30b26c182e88 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:27.430463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 865/1755] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:27.432261 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6d182-b734-42a8-8eb0-30b26c182e88 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:27.441153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-91b6d182-b734-42a8-8eb0-30b26c182e88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:27.467836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6d182-b734-42a8-8eb0-30b26c182e88 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:27.478887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc761418-f820-4ff8-ac8f-b1079cb1aabe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:27.479748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc761418-f820-4ff8-ac8f-b1079cb1aabe service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:27.480262 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc761418-f820-4ff8-ac8f-b1079cb1aabe service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:27.480683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc761418-f820-4ff8-ac8f-b1079cb1aabe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:27.497918 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf8953-ccfb-4123-bd9a-24644ce15f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:27.508050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf8953-ccfb-4123-bd9a-24644ce15f6e admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:27.508620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf8953-ccfb-4123-bd9a-24644ce15f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:27.509228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf8953-ccfb-4123-bd9a-24644ce15f6e admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:27.530365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf8953-ccfb-4123-bd9a-24644ce15f6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:27.532460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf8953-ccfb-4123-bd9a-24644ce15f6e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:27.614207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 891/1756] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:27 2022] POST /identity/v3/users => generated 282 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:27.642077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf8953-ccfb-4123-bd9a-24644ce15f6e admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:27.676711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf8953-ccfb-4123-bd9a-24644ce15f6e admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:27.690483 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53730b8-0f2e-4a2c-b476-2e353af3aab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:27.691343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53730b8-0f2e-4a2c-b476-2e353af3aab9 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:27.691976 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53730b8-0f2e-4a2c-b476-2e353af3aab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:27.693112 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53730b8-0f2e-4a2c-b476-2e353af3aab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:27.694343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53730b8-0f2e-4a2c-b476-2e353af3aab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:27.707227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53730b8-0f2e-4a2c-b476-2e353af3aab9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:27.721290 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc761418-f820-4ff8-ac8f-b1079cb1aabe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:27.723017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc761418-f820-4ff8-ac8f-b1079cb1aabe service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:27.728057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 892/1757] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:27.735565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53730b8-0f2e-4a2c-b476-2e353af3aab9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:27.750982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 866/1758] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:27 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:27.794794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53730b8-0f2e-4a2c-b476-2e353af3aab9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:27.823910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf3200-55d0-48ba-811d-dfaf09f1856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:27.824718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf3200-55d0-48ba-811d-dfaf09f1856f admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:27.825009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf3200-55d0-48ba-811d-dfaf09f1856f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:27.825459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf3200-55d0-48ba-811d-dfaf09f1856f admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:27.826858 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf3200-55d0-48ba-811d-dfaf09f1856f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:27.855296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf3200-55d0-48ba-811d-dfaf09f1856f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:27.906188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc761418-f820-4ff8-ac8f-b1079cb1aabe service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:27.912841 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 893/1759] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:27.917683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf3200-55d0-48ba-811d-dfaf09f1856f admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:27.923093 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc761418-f820-4ff8-ac8f-b1079cb1aabe service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:27.952884 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b605772-fde1-48f0-b406-bdd113d32082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:27.957985 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b605772-fde1-48f0-b406-bdd113d32082 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:27.958280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b605772-fde1-48f0-b406-bdd113d32082 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:27.963118 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b605772-fde1-48f0-b406-bdd113d32082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:27.970352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8b605772-fde1-48f0-b406-bdd113d32082 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:27.971285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf3200-55d0-48ba-811d-dfaf09f1856f admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:27.994940 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10351c99-9465-4077-bd80-59ac752b409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:28.002372 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10351c99-9465-4077-bd80-59ac752b409d admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:28.003100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10351c99-9465-4077-bd80-59ac752b409d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:28.003531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10351c99-9465-4077-bd80-59ac752b409d admin admin] PATH_INFO: `/v3/projects/48cda030737e42d48c101e9b5315fa36/users/90c6f467118d4ca9ad79f51231681995/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:28.074058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10351c99-9465-4077-bd80-59ac752b409d admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cda030737e42d48c101e9b5315fa36, user_id=90c6f467118d4ca9ad79f51231681995, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:28.077042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10351c99-9465-4077-bd80-59ac752b409d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:28.108217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 894/1760] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:27 2022] PUT /identity/v3/projects/48cda030737e42d48c101e9b5315fa36/users/90c6f467118d4ca9ad79f51231681995/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:28.122205 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507a97d-f257-4a78-a7eb-861f79334919 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:28.122501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507a97d-f257-4a78-a7eb-861f79334919 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:28.122784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507a97d-f257-4a78-a7eb-861f79334919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:28.202717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b605772-fde1-48f0-b406-bdd113d32082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:28.203008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-b507a97d-f257-4a78-a7eb-861f79334919 None None] MFA Rules not processed for user `90c6f467118d4ca9ad79f51231681995`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:28.205926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b507a97d-f257-4a78-a7eb-861f79334919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:28.207213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b605772-fde1-48f0-b406-bdd113d32082 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:28.236844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 867/1761] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:28.453651 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:28.475594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 895/1762] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:28 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:28.480022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b605772-fde1-48f0-b406-bdd113d32082 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:28.494070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b605772-fde1-48f0-b406-bdd113d32082 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:28.537738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4325871-c567-45ed-9467-19be11931fa3 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:28.538261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4325871-c567-45ed-9467-19be11931fa3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:28.538692 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4325871-c567-45ed-9467-19be11931fa3 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:28.540337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 896/1763] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:49:28.645925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191198d2-6be7-43e8-9ac9-35605fd03e35 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:28.647658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191198d2-6be7-43e8-9ac9-35605fd03e35 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:28.648568 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191198d2-6be7-43e8-9ac9-35605fd03e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:28.704360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:28.708389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-191198d2-6be7-43e8-9ac9-35605fd03e35 None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:28.709982 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-191198d2-6be7-43e8-9ac9-35605fd03e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:28.736883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30ce1d-29ff-410c-86bf-37fc92dec84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:28.737688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30ce1d-29ff-410c-86bf-37fc92dec84f admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:28.737893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30ce1d-29ff-410c-86bf-37fc92dec84f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:28.738167 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30ce1d-29ff-410c-86bf-37fc92dec84f admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:28.738867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30ce1d-29ff-410c-86bf-37fc92dec84f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:28.745844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30ce1d-29ff-410c-86bf-37fc92dec84f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:28.834065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 868/1764] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:28 2022] POST /identity/v3/projects => generated 363 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:28.838573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30ce1d-29ff-410c-86bf-37fc92dec84f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:28.904968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 897/1765] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:49:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 12 03:49:28.919280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191198d2-6be7-43e8-9ac9-35605fd03e35 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:28.920773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-191198d2-6be7-43e8-9ac9-35605fd03e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:28.970800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191198d2-6be7-43e8-9ac9-35605fd03e35 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:28.983527 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:28.984528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:28.984694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:28.985034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] PATH_INFO: `/v3/projects/3440ac7f7cb74b0292e846cdcd91c1d1` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:28.988264 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3440ac7f7cb74b0292e846cdcd91c1d1)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:28.992686 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30ce1d-29ff-410c-86bf-37fc92dec84f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:28.998319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:29.017230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb57ed3-1fc1-492e-a63f-228ba3c95c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:29.017746 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb57ed3-1fc1-492e-a63f-228ba3c95c56 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:29.018028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb57ed3-1fc1-492e-a63f-228ba3c95c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:29.018304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb57ed3-1fc1-492e-a63f-228ba3c95c56 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:29.019008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb57ed3-1fc1-492e-a63f-228ba3c95c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:29.027169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb57ed3-1fc1-492e-a63f-228ba3c95c56 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:29.064509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 869/1766] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:28 2022] POST /identity/v3/users => generated 288 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:29.066001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68071b2-90d7-420b-b15d-9e6e6ce609a8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:29.066454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68071b2-90d7-420b-b15d-9e6e6ce609a8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:29.066877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68071b2-90d7-420b-b15d-9e6e6ce609a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:29.133833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e68071b2-90d7-420b-b15d-9e6e6ce609a8 None None] MFA Rules not processed for user `bf67549bf7e443c6b77eaf71fb6b709c`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:29.138796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e68071b2-90d7-420b-b15d-9e6e6ce609a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:29.154511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3440ac7f7cb74b0292e846cdcd91c1d1', 'request_id': 'req-941b66d6-8496-4369-bbde-d728007beb6c'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:29.171785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3440ac7f7cb74b0292e846cdcd91c1d1', 'request_id': 'req-941b66d6-8496-4369-bbde-d728007beb6c'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:29.200245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3440ac7f7cb74b0292e846cdcd91c1d1', 'request_id': 'req-941b66d6-8496-4369-bbde-d728007beb6c'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:29.233530 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] The token cache is being invalidate because project 3440ac7f7cb74b0292e846cdcd91c1d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:29.233831 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:29.235335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:29.240847 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 898/1767] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:28 2022] DELETE /identity/v3/projects/3440ac7f7cb74b0292e846cdcd91c1d1 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:29.241809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:29.246040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:29.339378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 870/1768] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:29.340604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68071b2-90d7-420b-b15d-9e6e6ce609a8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:29.342488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e68071b2-90d7-420b-b15d-9e6e6ce609a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:29.444580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b66d6-8496-4369-bbde-d728007beb6c admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:29.466948 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69510fe2-1925-4fe2-8361-cb199587b800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:29.467798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69510fe2-1925-4fe2-8361-cb199587b800 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:29.468132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69510fe2-1925-4fe2-8361-cb199587b800 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:29.468416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69510fe2-1925-4fe2-8361-cb199587b800 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:29.469168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69510fe2-1925-4fe2-8361-cb199587b800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:29.476822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69510fe2-1925-4fe2-8361-cb199587b800 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:29.520584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 899/1769] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:29.521839 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81efc1-0e41-4aea-b6f2-630ab3b675cd None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:29.522130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81efc1-0e41-4aea-b6f2-630ab3b675cd None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:29.522409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81efc1-0e41-4aea-b6f2-630ab3b675cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:29.551941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:29.571758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-4a81efc1-0e41-4aea-b6f2-630ab3b675cd None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:29.573283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4a81efc1-0e41-4aea-b6f2-630ab3b675cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:29.600685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 900/1770] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:29.601689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81efc1-0e41-4aea-b6f2-630ab3b675cd None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:29.620571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81efc1-0e41-4aea-b6f2-630ab3b675cd None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:29.634402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8d7a9-f166-4c02-8e21-4da4f2d8b612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:29.635297 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8d7a9-f166-4c02-8e21-4da4f2d8b612 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:29.635591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8d7a9-f166-4c02-8e21-4da4f2d8b612 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:29.636015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8d7a9-f166-4c02-8e21-4da4f2d8b612 admin admin] PATH_INFO: `/v3/projects/6ead544a8a444568b7b683d6a1977c27/users/c28c03459b4648d3bc763b5ca8157811/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:29.637626 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68071b2-90d7-420b-b15d-9e6e6ce609a8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:29.670276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea70fc-1aea-47f5-b1d8-1a939527e473 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:29.671066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea70fc-1aea-47f5-b1d8-1a939527e473 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:29.671426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea70fc-1aea-47f5-b1d8-1a939527e473 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:29.671632 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea70fc-1aea-47f5-b1d8-1a939527e473 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:29.675863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8d7a9-f166-4c02-8e21-4da4f2d8b612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ead544a8a444568b7b683d6a1977c27, user_id=c28c03459b4648d3bc763b5ca8157811, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:29.678664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8d7a9-f166-4c02-8e21-4da4f2d8b612 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:29.681964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b5ea70fc-1aea-47f5-b1d8-1a939527e473 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:29.710763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 901/1771] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:29 2022] PUT /identity/v3/projects/6ead544a8a444568b7b683d6a1977c27/users/c28c03459b4648d3bc763b5ca8157811/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:29.712278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058624b2-79e5-46ce-b268-b159ca3b3eae None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:29.712571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058624b2-79e5-46ce-b268-b159ca3b3eae None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:29.712849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058624b2-79e5-46ce-b268-b159ca3b3eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:29.723147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:29.730797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea70fc-1aea-47f5-b1d8-1a939527e473 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:29.735152 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea70fc-1aea-47f5-b1d8-1a939527e473 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:29.773394 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 871/1772] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:29.774518 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea70fc-1aea-47f5-b1d8-1a939527e473 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:29.785491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-058624b2-79e5-46ce-b268-b159ca3b3eae None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:29.789968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea70fc-1aea-47f5-b1d8-1a939527e473 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:29.796624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-058624b2-79e5-46ce-b268-b159ca3b3eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:29.963339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ef781-f65e-4b83-844b-994a8fc06944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:29.964197 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ef781-f65e-4b83-844b-994a8fc06944 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:29.964628 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ef781-f65e-4b83-844b-994a8fc06944 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:29.965507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ef781-f65e-4b83-844b-994a8fc06944 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:29.966567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ef781-f65e-4b83-844b-994a8fc06944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:29.968534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59ef781-f65e-4b83-844b-994a8fc06944 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:30.005160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 872/1773] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:30.008938 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ef781-f65e-4b83-844b-994a8fc06944 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:30.031459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59ef781-f65e-4b83-844b-994a8fc06944 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:30.041498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc42ccf-e036-41bc-962b-d2c8fe61254a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:30.043067 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc42ccf-e036-41bc-962b-d2c8fe61254a admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:30.043809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc42ccf-e036-41bc-962b-d2c8fe61254a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:30.044418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc42ccf-e036-41bc-962b-d2c8fe61254a admin admin] PATH_INFO: `/v3/projects/6ead544a8a444568b7b683d6a1977c27/users/c28c03459b4648d3bc763b5ca8157811/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:30.065337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc42ccf-e036-41bc-962b-d2c8fe61254a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ead544a8a444568b7b683d6a1977c27, user_id=c28c03459b4648d3bc763b5ca8157811, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:30.069050 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc42ccf-e036-41bc-962b-d2c8fe61254a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:30.111521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 873/1774] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:30 2022] PUT /identity/v3/projects/6ead544a8a444568b7b683d6a1977c27/users/c28c03459b4648d3bc763b5ca8157811/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:30.114307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeae928-5646-4871-83ea-d3c9d503c32c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:30.114820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeae928-5646-4871-83ea-d3c9d503c32c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:30.115111 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeae928-5646-4871-83ea-d3c9d503c32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:30.154643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 902/1775] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:30.155697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058624b2-79e5-46ce-b268-b159ca3b3eae None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:30.212287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7aeae928-5646-4871-83ea-d3c9d503c32c None None] MFA Rules not processed for user `c28c03459b4648d3bc763b5ca8157811`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:30.217445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058624b2-79e5-46ce-b268-b159ca3b3eae None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:30.219266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7aeae928-5646-4871-83ea-d3c9d503c32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:30.438892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c6966-774c-4851-bd8b-561fda75699d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:30.440233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c6966-774c-4851-bd8b-561fda75699d admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:30.440677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c6966-774c-4851-bd8b-561fda75699d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:30.441302 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c6966-774c-4851-bd8b-561fda75699d admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:30.442462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c6966-774c-4851-bd8b-561fda75699d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:30.443076 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c6966-774c-4851-bd8b-561fda75699d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:30.446420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c6966-774c-4851-bd8b-561fda75699d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:30.459204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 874/1776] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:30 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:30.467865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 903/1777] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:30.472710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeae928-5646-4871-83ea-d3c9d503c32c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:30.500604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeae928-5646-4871-83ea-d3c9d503c32c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:30.509690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c6966-774c-4851-bd8b-561fda75699d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:30.514466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-218c6966-774c-4851-bd8b-561fda75699d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:30.531452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7808b-d192-40a1-9e90-b82c8e0ffed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:30.533860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7808b-d192-40a1-9e90-b82c8e0ffed4 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:30.535403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7808b-d192-40a1-9e90-b82c8e0ffed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:30.536264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7808b-d192-40a1-9e90-b82c8e0ffed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:30.538457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7808b-d192-40a1-9e90-b82c8e0ffed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:30.541921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7808b-d192-40a1-9e90-b82c8e0ffed4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:30.607333 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 875/1778] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:30 2022] POST /identity/v3/projects => generated 403 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:30.614518 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7808b-d192-40a1-9e90-b82c8e0ffed4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:30.670059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c6966-774c-4851-bd8b-561fda75699d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:30.736593 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:30.739100 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:30.802342 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7808b-d192-40a1-9e90-b82c8e0ffed4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:30.825241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe37b40-07c7-4473-8ff0-30982d4123c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:30.825955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe37b40-07c7-4473-8ff0-30982d4123c0 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:30.826241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe37b40-07c7-4473-8ff0-30982d4123c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:30.826518 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe37b40-07c7-4473-8ff0-30982d4123c0 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:30.827230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe37b40-07c7-4473-8ff0-30982d4123c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:30.837705 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe37b40-07c7-4473-8ff0-30982d4123c0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:30.841556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc04bd6-35c2-44f6-b2fa-a5836d429af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:30.841896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc04bd6-35c2-44f6-b2fa-a5836d429af1 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:30.842221 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc04bd6-35c2-44f6-b2fa-a5836d429af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:30.842457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc04bd6-35c2-44f6-b2fa-a5836d429af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:30.948096 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 876/1779] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:30 2022] POST /identity/v3/users => generated 302 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:30.960281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe37b40-07c7-4473-8ff0-30982d4123c0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:31.005847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe37b40-07c7-4473-8ff0-30982d4123c0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:31.030501 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740b87f-ffc0-4fe4-b5a3-5807d86b3375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:31.041805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740b87f-ffc0-4fe4-b5a3-5807d86b3375 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:31.043041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740b87f-ffc0-4fe4-b5a3-5807d86b3375 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:31.047649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740b87f-ffc0-4fe4-b5a3-5807d86b3375 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:31.050187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740b87f-ffc0-4fe4-b5a3-5807d86b3375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:31.052671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc04bd6-35c2-44f6-b2fa-a5836d429af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:31.054128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740b87f-ffc0-4fe4-b5a3-5807d86b3375 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:31.063157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc04bd6-35c2-44f6-b2fa-a5836d429af1 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:31.096990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 877/1780] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:31.110210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 904/1781] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:30 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:31.111192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740b87f-ffc0-4fe4-b5a3-5807d86b3375 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:31.163786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740b87f-ffc0-4fe4-b5a3-5807d86b3375 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:31.181071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973e172-78ed-4fb3-ab50-02368f0e5c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:31.185441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973e172-78ed-4fb3-ab50-02368f0e5c57 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:31.185924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973e172-78ed-4fb3-ab50-02368f0e5c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:31.186406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973e172-78ed-4fb3-ab50-02368f0e5c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:31.187516 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973e172-78ed-4fb3-ab50-02368f0e5c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:31.189742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973e172-78ed-4fb3-ab50-02368f0e5c57 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:31.229086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 878/1782] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:31.233162 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc04bd6-35c2-44f6-b2fa-a5836d429af1 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:31.252260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc04bd6-35c2-44f6-b2fa-a5836d429af1 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:31.263837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215085ff-ff30-4f31-854f-e896a3c74a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:31.266559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215085ff-ff30-4f31-854f-e896a3c74a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:31.267076 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215085ff-ff30-4f31-854f-e896a3c74a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:31.268315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215085ff-ff30-4f31-854f-e896a3c74a85 admin admin] PATH_INFO: `/v3/projects/5683619f4ab7459bb5cc12d91703b28d/users/dba7c0a9983946089fd66c1f3b12f327/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:31.322589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215085ff-ff30-4f31-854f-e896a3c74a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5683619f4ab7459bb5cc12d91703b28d, user_id=dba7c0a9983946089fd66c1f3b12f327, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:31.326134 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215085ff-ff30-4f31-854f-e896a3c74a85 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:31.380741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 905/1783] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:31 2022] PUT /identity/v3/projects/5683619f4ab7459bb5cc12d91703b28d/users/dba7c0a9983946089fd66c1f3b12f327/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:31.385710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6731c7-b9a8-4bea-9682-0a1ae249b7cf None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:31.386195 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6731c7-b9a8-4bea-9682-0a1ae249b7cf None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:31.386474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6731c7-b9a8-4bea-9682-0a1ae249b7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:31.433614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3b6731c7-b9a8-4bea-9682-0a1ae249b7cf None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:31.434846 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3b6731c7-b9a8-4bea-9682-0a1ae249b7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:31.507270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 879/1784] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:31.658618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215085ff-ff30-4f31-854f-e896a3c74a85 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:31.662919 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-215085ff-ff30-4f31-854f-e896a3c74a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:31.868412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215085ff-ff30-4f31-854f-e896a3c74a85 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:31.879955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bb34a-32ec-4657-b708-ee589edc9968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:31.880937 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bb34a-32ec-4657-b708-ee589edc9968 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:31.881482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bb34a-32ec-4657-b708-ee589edc9968 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:31.881826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bb34a-32ec-4657-b708-ee589edc9968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:31.929465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bb34a-32ec-4657-b708-ee589edc9968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:31.931074 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bb34a-32ec-4657-b708-ee589edc9968 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:31.966131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 906/1785] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:32.009219 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6731c7-b9a8-4bea-9682-0a1ae249b7cf None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:32.243531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6731c7-b9a8-4bea-9682-0a1ae249b7cf None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:32.249850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f7481-4ff2-4912-b775-39cf738889be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:32.257145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f7481-4ff2-4912-b775-39cf738889be service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:32.257145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f7481-4ff2-4912-b775-39cf738889be service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:32.257145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f7481-4ff2-4912-b775-39cf738889be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:32.259370 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1d6f7481-4ff2-4912-b775-39cf738889be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:32.451876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f7481-4ff2-4912-b775-39cf738889be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:32.453434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f7481-4ff2-4912-b775-39cf738889be service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:32.485525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 880/1786] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:32 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:33.398789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f18e9-bce2-46d1-b9bd-4da3d76379c7 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:33.399798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f18e9-bce2-46d1-b9bd-4da3d76379c7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:33.400362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f18e9-bce2-46d1-b9bd-4da3d76379c7 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:33.402509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 907/1787] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:49:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:49:33.445590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7963ee6-c15f-4735-ab83-dcd942ed5db7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:33.446094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7963ee6-c15f-4735-ab83-dcd942ed5db7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:33.446310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7963ee6-c15f-4735-ab83-dcd942ed5db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:33.456035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:33.458013 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f18e9-bce2-46d1-b9bd-4da3d76379c7 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:33.459968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0e0f18e9-bce2-46d1-b9bd-4da3d76379c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:33.502365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f18e9-bce2-46d1-b9bd-4da3d76379c7 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:33.524972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54138912-8370-48cd-8ec0-55ad40d083e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:33.526136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54138912-8370-48cd-8ec0-55ad40d083e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:33.526594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54138912-8370-48cd-8ec0-55ad40d083e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:33.527018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54138912-8370-48cd-8ec0-55ad40d083e1 admin admin] PATH_INFO: `/v3/users/4448db1fb05841e88e527e44c4a23b3f` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:33.533413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54138912-8370-48cd-8ec0-55ad40d083e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4448db1fb05841e88e527e44c4a23b3f)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:33.534683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: WARNING py.warnings [None req-54138912-8370-48cd-8ec0-55ad40d083e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 03:49:33.534683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: warnings.warn(msg) Jul 12 03:49:33.534683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: Jul 12 03:49:33.541993 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54138912-8370-48cd-8ec0-55ad40d083e1 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:33.542421 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-f7963ee6-c15f-4735-ab83-dcd942ed5db7 None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:33.544142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f7963ee6-c15f-4735-ab83-dcd942ed5db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:33.711579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-54138912-8370-48cd-8ec0-55ad40d083e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4448db1fb05841e88e527e44c4a23b3f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:33.712302 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-54138912-8370-48cd-8ec0-55ad40d083e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4448db1fb05841e88e527e44c4a23b3f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:33.715667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:33.728355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-54138912-8370-48cd-8ec0-55ad40d083e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4448db1fb05841e88e527e44c4a23b3f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:33.755509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-54138912-8370-48cd-8ec0-55ad40d083e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4448db1fb05841e88e527e44c4a23b3f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:33.804276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-54138912-8370-48cd-8ec0-55ad40d083e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4448db1fb05841e88e527e44c4a23b3f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:33.822528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 908/1788] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:33 2022] DELETE /identity/v3/users/4448db1fb05841e88e527e44c4a23b3f => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:33.824098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f369151-cc03-4d1c-b36b-3ad9f0a78d47 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:33.830844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f369151-cc03-4d1c-b36b-3ad9f0a78d47 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:33.834048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f369151-cc03-4d1c-b36b-3ad9f0a78d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:33.849234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 881/1789] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:49:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:33.868323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-7f369151-cc03-4d1c-b36b-3ad9f0a78d47 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:33.869600 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7f369151-cc03-4d1c-b36b-3ad9f0a78d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:33.924365 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7963ee6-c15f-4735-ab83-dcd942ed5db7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:33.926629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f7963ee6-c15f-4735-ab83-dcd942ed5db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:33.967576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7963ee6-c15f-4735-ab83-dcd942ed5db7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:34.056268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 909/1790] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:34.057438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f369151-cc03-4d1c-b36b-3ad9f0a78d47 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:34.068867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7f369151-cc03-4d1c-b36b-3ad9f0a78d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:34.095591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f369151-cc03-4d1c-b36b-3ad9f0a78d47 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:34.109140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa909aa-2297-4ba0-b994-9ba7713dd253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:34.110607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cba64e-a09d-4b97-a9c9-ae1979c0e9de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:34.111510 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cba64e-a09d-4b97-a9c9-ae1979c0e9de service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:34.111652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa909aa-2297-4ba0-b994-9ba7713dd253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:34.111986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cba64e-a09d-4b97-a9c9-ae1979c0e9de service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:34.112116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa909aa-2297-4ba0-b994-9ba7713dd253 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:34.112367 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa909aa-2297-4ba0-b994-9ba7713dd253 admin admin] PATH_INFO: `/v3/users/7daefac2eadd4eeea21d2d5a5e2d01db` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:34.112612 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cba64e-a09d-4b97-a9c9-ae1979c0e9de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:34.115356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa909aa-2297-4ba0-b994-9ba7713dd253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7daefac2eadd4eeea21d2d5a5e2d01db)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:34.120090 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa909aa-2297-4ba0-b994-9ba7713dd253 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:34.129545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cba64e-a09d-4b97-a9c9-ae1979c0e9de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:34.132045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cba64e-a09d-4b97-a9c9-ae1979c0e9de service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:34.180329 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 882/1791] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:34.181344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cba64e-a09d-4b97-a9c9-ae1979c0e9de service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:34.189287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-62cba64e-a09d-4b97-a9c9-ae1979c0e9de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:34.233818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5fa909aa-2297-4ba0-b994-9ba7713dd253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7daefac2eadd4eeea21d2d5a5e2d01db'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:34.234968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5fa909aa-2297-4ba0-b994-9ba7713dd253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7daefac2eadd4eeea21d2d5a5e2d01db'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:34.251688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5fa909aa-2297-4ba0-b994-9ba7713dd253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7daefac2eadd4eeea21d2d5a5e2d01db'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:34.274276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5fa909aa-2297-4ba0-b994-9ba7713dd253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7daefac2eadd4eeea21d2d5a5e2d01db'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:34.298532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5fa909aa-2297-4ba0-b994-9ba7713dd253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7daefac2eadd4eeea21d2d5a5e2d01db'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:34.315118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 910/1792] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:34 2022] DELETE /identity/v3/users/7daefac2eadd4eeea21d2d5a5e2d01db => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:34.317156 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378352d5-517c-403a-8ed1-cd781b46b040 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:34.317357 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378352d5-517c-403a-8ed1-cd781b46b040 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:34.317779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378352d5-517c-403a-8ed1-cd781b46b040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:34.318168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cba64e-a09d-4b97-a9c9-ae1979c0e9de service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:34.345917 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-378352d5-517c-403a-8ed1-cd781b46b040 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:34.350499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-378352d5-517c-403a-8ed1-cd781b46b040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:34.413054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:34.441506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:34.461377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281f918-27bd-459e-b4b3-6d98c835fc6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:34.462058 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281f918-27bd-459e-b4b3-6d98c835fc6c service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:34.462516 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281f918-27bd-459e-b4b3-6d98c835fc6c service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:34.462798 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281f918-27bd-459e-b4b3-6d98c835fc6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:34.469841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e281f918-27bd-459e-b4b3-6d98c835fc6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:34.482617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 911/1793] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:34.509313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378352d5-517c-403a-8ed1-cd781b46b040 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:34.512383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-378352d5-517c-403a-8ed1-cd781b46b040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:34.540158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378352d5-517c-403a-8ed1-cd781b46b040 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:34.550650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13177a4d-e6e0-4ec6-87e9-7a143b27cbfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:34.553538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13177a4d-e6e0-4ec6-87e9-7a143b27cbfe service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:34.554014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13177a4d-e6e0-4ec6-87e9-7a143b27cbfe service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:34.555731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13177a4d-e6e0-4ec6-87e9-7a143b27cbfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:34.572830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13177a4d-e6e0-4ec6-87e9-7a143b27cbfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:34.574955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13177a4d-e6e0-4ec6-87e9-7a143b27cbfe service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:34.588752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 912/1794] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:49:34.601114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281f918-27bd-459e-b4b3-6d98c835fc6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:34.603401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281f918-27bd-459e-b4b3-6d98c835fc6c service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:34.622317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 883/1795] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:35.192827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e08b27-5644-4554-acb6-392b46b64fb1 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:35.193132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e08b27-5644-4554-acb6-392b46b64fb1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:35.193421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e08b27-5644-4554-acb6-392b46b64fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:35.221599 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281f918-27bd-459e-b4b3-6d98c835fc6c service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:35.223304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e281f918-27bd-459e-b4b3-6d98c835fc6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:35.239401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281f918-27bd-459e-b4b3-6d98c835fc6c service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:35.244409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:35.245288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:35.245753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:35.246033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] PATH_INFO: `/v3/projects/be14eac72d2342bca5a603236e4441b2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:35.247424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be14eac72d2342bca5a603236e4441b2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:35.249387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:35.273667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-84e08b27-5644-4554-acb6-392b46b64fb1 None None] MFA Rules not processed for user `dba7c0a9983946089fd66c1f3b12f327`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:35.275469 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-84e08b27-5644-4554-acb6-392b46b64fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:35.308234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be14eac72d2342bca5a603236e4441b2', 'request_id': 'req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.318847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be14eac72d2342bca5a603236e4441b2', 'request_id': 'req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.329438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be14eac72d2342bca5a603236e4441b2', 'request_id': 'req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.344182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] The token cache is being invalidate because project be14eac72d2342bca5a603236e4441b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:35.344481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.345160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.346782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 884/1796] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:35 2022] DELETE /identity/v3/projects/be14eac72d2342bca5a603236e4441b2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:35.348242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:35.349197 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:35.368795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 913/1797] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:35.370115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e08b27-5644-4554-acb6-392b46b64fb1 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:35.371462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-84e08b27-5644-4554-acb6-392b46b64fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:35.386189 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e08b27-5644-4554-acb6-392b46b64fb1 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:35.421173 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5796dea-c8b7-4718-b2cb-864cdc02aaa9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:35.425833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:35.426764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:35.427058 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:35.427330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] PATH_INFO: `/v3/projects/93c8aaa9f9a64f7d8cbbd40054419fb2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:35.428556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c8aaa9f9a64f7d8cbbd40054419fb2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:35.430519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:35.441973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:35.442699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:35.443633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:35.443920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:35.444261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] PATH_INFO: `/v3/users/14d996c7e2124ad68232be116dd821f0` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:35.445596 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d996c7e2124ad68232be116dd821f0)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:35.447536 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:35.483514 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c8aaa9f9a64f7d8cbbd40054419fb2', 'request_id': 'req-9e5a2d28-2611-43dd-b941-3641ced4ba48'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.490933 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c8aaa9f9a64f7d8cbbd40054419fb2', 'request_id': 'req-9e5a2d28-2611-43dd-b941-3641ced4ba48'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.496917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c8aaa9f9a64f7d8cbbd40054419fb2', 'request_id': 'req-9e5a2d28-2611-43dd-b941-3641ced4ba48'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.497072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d996c7e2124ad68232be116dd821f0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.497585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d996c7e2124ad68232be116dd821f0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.505727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d996c7e2124ad68232be116dd821f0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.509530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] The token cache is being invalidate because project 93c8aaa9f9a64f7d8cbbd40054419fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:35.509890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.510486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:35.510651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.512744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 885/1798] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:35 2022] DELETE /identity/v3/projects/93c8aaa9f9a64f7d8cbbd40054419fb2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:35.513595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:35.513706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d996c7e2124ad68232be116dd821f0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.514897 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:35.527844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d996c7e2124ad68232be116dd821f0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.534421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 914/1799] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:35 2022] DELETE /identity/v3/users/14d996c7e2124ad68232be116dd821f0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:35.535499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:35.536677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:35.586442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a2d28-2611-43dd-b941-3641ced4ba48 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:35.632112 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa10d47-b07b-4299-af28-c728ee9b8a2d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:35.637394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:35.638236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:35.638526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:35.641511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] PATH_INFO: `/v3/users/3151bfdb62ae4824b5bfa1de006ac4de` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:35.642445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3151bfdb62ae4824b5bfa1de006ac4de)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:35.645209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:35.672343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876253c-ba17-4b2f-ac3c-8cee873910f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:35.673032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876253c-ba17-4b2f-ac3c-8cee873910f2 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:35.674123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876253c-ba17-4b2f-ac3c-8cee873910f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:35.674412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876253c-ba17-4b2f-ac3c-8cee873910f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:35.675121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876253c-ba17-4b2f-ac3c-8cee873910f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:35.678110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876253c-ba17-4b2f-ac3c-8cee873910f2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:35.713813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3151bfdb62ae4824b5bfa1de006ac4de'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.714880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3151bfdb62ae4824b5bfa1de006ac4de'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.725756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 886/1800] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:35 2022] POST /identity/v3/projects => generated 405 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:35.730023 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876253c-ba17-4b2f-ac3c-8cee873910f2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:35.730194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3151bfdb62ae4824b5bfa1de006ac4de'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.731725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e876253c-ba17-4b2f-ac3c-8cee873910f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:35.741697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3151bfdb62ae4824b5bfa1de006ac4de'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.762320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3151bfdb62ae4824b5bfa1de006ac4de'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:35.770724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 915/1801] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:35 2022] DELETE /identity/v3/users/3151bfdb62ae4824b5bfa1de006ac4de => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:35.779271 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:35.830300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876253c-ba17-4b2f-ac3c-8cee873910f2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:35.897841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea2df4-966d-47b4-af7a-1665fd5c9214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:35.898454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea2df4-966d-47b4-af7a-1665fd5c9214 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:35.898727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea2df4-966d-47b4-af7a-1665fd5c9214 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:35.899056 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea2df4-966d-47b4-af7a-1665fd5c9214 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:35.899709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea2df4-966d-47b4-af7a-1665fd5c9214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:35.901712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea2df4-966d-47b4-af7a-1665fd5c9214 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:35.936059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 887/1802] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:35 2022] POST /identity/v3/users => generated 309 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:35.939168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:35.951228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43879432-aa58-454b-afe7-565a8e3d55f7 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:35.955826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb347f-a8e7-4c26-a9cb-48736b2d3959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:35.961989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb347f-a8e7-4c26-a9cb-48736b2d3959 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:35.961989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb347f-a8e7-4c26-a9cb-48736b2d3959 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:35.961989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb347f-a8e7-4c26-a9cb-48736b2d3959 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:35.961989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb347f-a8e7-4c26-a9cb-48736b2d3959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:35.961989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb347f-a8e7-4c26-a9cb-48736b2d3959 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:35.973460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 916/1803] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:35.974750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea2df4-966d-47b4-af7a-1665fd5c9214 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:35.988926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea2df4-966d-47b4-af7a-1665fd5c9214 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:35.994380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd324c8-ec75-48ab-907d-6623605c1eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:35.995465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd324c8-ec75-48ab-907d-6623605c1eff admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:35.995745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd324c8-ec75-48ab-907d-6623605c1eff admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:35.996506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd324c8-ec75-48ab-907d-6623605c1eff admin admin] PATH_INFO: `/v3/projects/37203b7ae9cd41ebbb14b6852c817d38/users/6938cfb82a8e4a458b2e62a520f16eb4/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.012604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd324c8-ec75-48ab-907d-6623605c1eff admin admin] RBAC: Authorizing `identity:create_grant(project_id=37203b7ae9cd41ebbb14b6852c817d38, user_id=6938cfb82a8e4a458b2e62a520f16eb4, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:36.015303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd324c8-ec75-48ab-907d-6623605c1eff admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:36.036827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 888/1804] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:35 2022] PUT /identity/v3/projects/37203b7ae9cd41ebbb14b6852c817d38/users/6938cfb82a8e4a458b2e62a520f16eb4/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:36.040674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb347f-a8e7-4c26-a9cb-48736b2d3959 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:36.051717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb347f-a8e7-4c26-a9cb-48736b2d3959 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:36.072383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd324c8-ec75-48ab-907d-6623605c1eff admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:36.074719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3dd324c8-ec75-48ab-907d-6623605c1eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:36.107293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:36.127658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27d764-c2f1-4d8a-b17c-d09c7c34dadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:36.128366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27d764-c2f1-4d8a-b17c-d09c7c34dadb admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.128650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27d764-c2f1-4d8a-b17c-d09c7c34dadb admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.128928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27d764-c2f1-4d8a-b17c-d09c7c34dadb admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.129662 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27d764-c2f1-4d8a-b17c-d09c7c34dadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:36.131346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27d764-c2f1-4d8a-b17c-d09c7c34dadb admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:36.144706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 917/1805] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:49:36.149342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27d764-c2f1-4d8a-b17c-d09c7c34dadb admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:36.160586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27d764-c2f1-4d8a-b17c-d09c7c34dadb admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:36.165482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbb9e8-5624-46db-890c-ede271af4406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:36.166453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbb9e8-5624-46db-890c-ede271af4406 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.166591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd324c8-ec75-48ab-907d-6623605c1eff admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:36.166692 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbb9e8-5624-46db-890c-ede271af4406 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.166981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbb9e8-5624-46db-890c-ede271af4406 admin admin] PATH_INFO: `/v3/projects/37203b7ae9cd41ebbb14b6852c817d38/users/6938cfb82a8e4a458b2e62a520f16eb4/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.169019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbb9e8-5624-46db-890c-ede271af4406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37203b7ae9cd41ebbb14b6852c817d38, user_id=6938cfb82a8e4a458b2e62a520f16eb4, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:36.171564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175ff26-ef45-40b8-b8de-1d51de9eb676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:36.171832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbb9e8-5624-46db-890c-ede271af4406 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:36.172475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175ff26-ef45-40b8-b8de-1d51de9eb676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.172756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175ff26-ef45-40b8-b8de-1d51de9eb676 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.173030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175ff26-ef45-40b8-b8de-1d51de9eb676 admin admin] PATH_INFO: `/v3/projects/9e3717707fa149efbf9d59296901cbd2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.174463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175ff26-ef45-40b8-b8de-1d51de9eb676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e3717707fa149efbf9d59296901cbd2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:36.176544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175ff26-ef45-40b8-b8de-1d51de9eb676 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:36.186619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 918/1806] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:36 2022] PUT /identity/v3/projects/37203b7ae9cd41ebbb14b6852c817d38/users/6938cfb82a8e4a458b2e62a520f16eb4/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:36.187321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbb9e8-5624-46db-890c-ede271af4406 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:36.188912 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-49fbb9e8-5624-46db-890c-ede271af4406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:36.261446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:36.263562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9175ff26-ef45-40b8-b8de-1d51de9eb676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e3717707fa149efbf9d59296901cbd2', 'request_id': 'req-9175ff26-ef45-40b8-b8de-1d51de9eb676'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:36.282810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9175ff26-ef45-40b8-b8de-1d51de9eb676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e3717707fa149efbf9d59296901cbd2', 'request_id': 'req-9175ff26-ef45-40b8-b8de-1d51de9eb676'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:36.291717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9175ff26-ef45-40b8-b8de-1d51de9eb676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e3717707fa149efbf9d59296901cbd2', 'request_id': 'req-9175ff26-ef45-40b8-b8de-1d51de9eb676'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:36.318537 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9175ff26-ef45-40b8-b8de-1d51de9eb676 admin admin] The token cache is being invalidate because project 9e3717707fa149efbf9d59296901cbd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:36.318762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbb9e8-5624-46db-890c-ede271af4406 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:36.318857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9175ff26-ef45-40b8-b8de-1d51de9eb676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:36.319593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9175ff26-ef45-40b8-b8de-1d51de9eb676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:36.321383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 889/1807] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:36 2022] DELETE /identity/v3/projects/9e3717707fa149efbf9d59296901cbd2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:36.326469 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eb8db-69e1-40e2-805f-4458d5e16a3a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.326643 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eb8db-69e1-40e2-805f-4458d5e16a3a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.326787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eb8db-69e1-40e2-805f-4458d5e16a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.357770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2d2eb8db-69e1-40e2-805f-4458d5e16a3a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:36.359851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2d2eb8db-69e1-40e2-805f-4458d5e16a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:36.404893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15ef3d-e535-4301-92b9-b7d4263c5573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:36.406072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15ef3d-e535-4301-92b9-b7d4263c5573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.406371 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15ef3d-e535-4301-92b9-b7d4263c5573 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.406634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15ef3d-e535-4301-92b9-b7d4263c5573 admin admin] PATH_INFO: `/v3/projects/2a8a9a1c957746c58fdc77099580a090` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.407890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15ef3d-e535-4301-92b9-b7d4263c5573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a8a9a1c957746c58fdc77099580a090)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:36.410936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15ef3d-e535-4301-92b9-b7d4263c5573 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:36.454082 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 890/1808] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:36.463360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e16f78-d883-4798-b9bf-2787b22a6a8e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.463360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e16f78-d883-4798-b9bf-2787b22a6a8e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.463673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e16f78-d883-4798-b9bf-2787b22a6a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.476223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8a15ef3d-e535-4301-92b9-b7d4263c5573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a8a9a1c957746c58fdc77099580a090', 'request_id': 'req-8a15ef3d-e535-4301-92b9-b7d4263c5573'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:36.482723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-72e16f78-d883-4798-b9bf-2787b22a6a8e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:36.484110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-72e16f78-d883-4798-b9bf-2787b22a6a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:36.487287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8a15ef3d-e535-4301-92b9-b7d4263c5573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a8a9a1c957746c58fdc77099580a090', 'request_id': 'req-8a15ef3d-e535-4301-92b9-b7d4263c5573'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:36.496458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8a15ef3d-e535-4301-92b9-b7d4263c5573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a8a9a1c957746c58fdc77099580a090', 'request_id': 'req-8a15ef3d-e535-4301-92b9-b7d4263c5573'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:36.510156 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8a15ef3d-e535-4301-92b9-b7d4263c5573 admin admin] The token cache is being invalidate because project 2a8a9a1c957746c58fdc77099580a090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:36.510453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8a15ef3d-e535-4301-92b9-b7d4263c5573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:36.511145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8a15ef3d-e535-4301-92b9-b7d4263c5573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:36.513125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 919/1809] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:36 2022] DELETE /identity/v3/projects/2a8a9a1c957746c58fdc77099580a090 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:36.517970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decafbf6-82c3-4b5c-bf40-215c6b1ca215 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.517970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decafbf6-82c3-4b5c-bf40-215c6b1ca215 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.518193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decafbf6-82c3-4b5c-bf40-215c6b1ca215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.527014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:36.539081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-decafbf6-82c3-4b5c-bf40-215c6b1ca215 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:36.543930 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-decafbf6-82c3-4b5c-bf40-215c6b1ca215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:36.563443 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 891/1810] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:36.567563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e16f78-d883-4798-b9bf-2787b22a6a8e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:36.577749 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e16f78-d883-4798-b9bf-2787b22a6a8e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:36.583337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da23869-46d8-4ca0-a0fa-73c4f37e3d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:36.584408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da23869-46d8-4ca0-a0fa-73c4f37e3d0b admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.584834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da23869-46d8-4ca0-a0fa-73c4f37e3d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.585265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da23869-46d8-4ca0-a0fa-73c4f37e3d0b admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.586135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da23869-46d8-4ca0-a0fa-73c4f37e3d0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:36.586590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da23869-46d8-4ca0-a0fa-73c4f37e3d0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:36.588413 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da23869-46d8-4ca0-a0fa-73c4f37e3d0b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:36.607817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 892/1811] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:36.613293 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da23869-46d8-4ca0-a0fa-73c4f37e3d0b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:36.616055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 920/1812] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:36.623789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da5755-fe15-468f-a618-ec90c88a9a1f None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.624436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da5755-fe15-468f-a618-ec90c88a9a1f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.624854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da5755-fe15-468f-a618-ec90c88a9a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.624961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da23869-46d8-4ca0-a0fa-73c4f37e3d0b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:36.629843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029f6a6-a194-4503-a101-fce3fc7aa413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:36.630571 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029f6a6-a194-4503-a101-fce3fc7aa413 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.631033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029f6a6-a194-4503-a101-fce3fc7aa413 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.631315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029f6a6-a194-4503-a101-fce3fc7aa413 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.632070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029f6a6-a194-4503-a101-fce3fc7aa413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:36.635352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029f6a6-a194-4503-a101-fce3fc7aa413 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:36.646698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-03da5755-fe15-468f-a618-ec90c88a9a1f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:36.648171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-03da5755-fe15-468f-a618-ec90c88a9a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:36.657023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 921/1813] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:36.661377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da5755-fe15-468f-a618-ec90c88a9a1f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:36.673708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 893/1814] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:36 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:36.673886 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da5755-fe15-468f-a618-ec90c88a9a1f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:36.674812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029f6a6-a194-4503-a101-fce3fc7aa413 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:36.728923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:36.748770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58327dd-a3c7-4a03-ae5e-06c26f8a2ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:36.749351 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58327dd-a3c7-4a03-ae5e-06c26f8a2ddc admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.749705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58327dd-a3c7-4a03-ae5e-06c26f8a2ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.750062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58327dd-a3c7-4a03-ae5e-06c26f8a2ddc admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.750898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58327dd-a3c7-4a03-ae5e-06c26f8a2ddc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:36.751238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58327dd-a3c7-4a03-ae5e-06c26f8a2ddc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:36.752597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029f6a6-a194-4503-a101-fce3fc7aa413 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:36.752902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58327dd-a3c7-4a03-ae5e-06c26f8a2ddc admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:36.759378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32788e-6d20-4daa-9e96-5d634851b3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:36.760162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32788e-6d20-4daa-9e96-5d634851b3f2 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.760527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32788e-6d20-4daa-9e96-5d634851b3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.760724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32788e-6d20-4daa-9e96-5d634851b3f2 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.761872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32788e-6d20-4daa-9e96-5d634851b3f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:36.763638 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32788e-6d20-4daa-9e96-5d634851b3f2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:36.767861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 922/1815] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:49:36.773255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58327dd-a3c7-4a03-ae5e-06c26f8a2ddc admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:36.786701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58327dd-a3c7-4a03-ae5e-06c26f8a2ddc admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:36.786916 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 894/1816] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:36 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:36.792383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef73dd-8d28-41cf-9e6e-1e14fcce86a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:36.793053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef73dd-8d28-41cf-9e6e-1e14fcce86a6 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.793339 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef73dd-8d28-41cf-9e6e-1e14fcce86a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.793618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef73dd-8d28-41cf-9e6e-1e14fcce86a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.794308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef73dd-8d28-41cf-9e6e-1e14fcce86a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:36.794412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32788e-6d20-4daa-9e96-5d634851b3f2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:36.796267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef73dd-8d28-41cf-9e6e-1e14fcce86a6 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:36.805653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32788e-6d20-4daa-9e96-5d634851b3f2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:36.811425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497da2d-2e94-4427-b9e1-488522a518ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:36.812198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497da2d-2e94-4427-b9e1-488522a518ab admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.812576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497da2d-2e94-4427-b9e1-488522a518ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.812678 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497da2d-2e94-4427-b9e1-488522a518ab admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.813585 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497da2d-2e94-4427-b9e1-488522a518ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:36.815363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497da2d-2e94-4427-b9e1-488522a518ab admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:36.826861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 923/1817] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:36 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:36.829342 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 895/1818] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:36.833522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef73dd-8d28-41cf-9e6e-1e14fcce86a6 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:36.833699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497da2d-2e94-4427-b9e1-488522a518ab admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:36.871464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497da2d-2e94-4427-b9e1-488522a518ab admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:36.974551 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19f54-976e-4ef3-ba01-d79bcdc459f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:36.975557 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19f54-976e-4ef3-ba01-d79bcdc459f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.975787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef73dd-8d28-41cf-9e6e-1e14fcce86a6 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:36.976220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19f54-976e-4ef3-ba01-d79bcdc459f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.976664 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19f54-976e-4ef3-ba01-d79bcdc459f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:36.977826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee19f54-976e-4ef3-ba01-d79bcdc459f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:36.980291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee19f54-976e-4ef3-ba01-d79bcdc459f3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:36.993687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 896/1819] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:36.996536 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57696b0-9f13-4a10-82bf-12cbfdf0a161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:36.997276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57696b0-9f13-4a10-82bf-12cbfdf0a161 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:36.997541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57696b0-9f13-4a10-82bf-12cbfdf0a161 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:36.998290 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19f54-976e-4ef3-ba01-d79bcdc459f3 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:37.000201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57696b0-9f13-4a10-82bf-12cbfdf0a161 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.000962 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57696b0-9f13-4a10-82bf-12cbfdf0a161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:37.006603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57696b0-9f13-4a10-82bf-12cbfdf0a161 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:37.010243 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19f54-976e-4ef3-ba01-d79bcdc459f3 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:37.015883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a4644-add3-449c-9bdc-55d277c82d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:37.016908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a4644-add3-449c-9bdc-55d277c82d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.018262 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a4644-add3-449c-9bdc-55d277c82d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.018262 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a4644-add3-449c-9bdc-55d277c82d9e admin admin] PATH_INFO: `/v3/projects/fdc613edd261411fa7c9352fd2d0285e/users/1fbaf5030dfa409bb52b7d0b737bdb99/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.040332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a4644-add3-449c-9bdc-55d277c82d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc613edd261411fa7c9352fd2d0285e, user_id=1fbaf5030dfa409bb52b7d0b737bdb99, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:37.042982 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:37.044956 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a4644-add3-449c-9bdc-55d277c82d9e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:37.052423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 924/1820] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:36 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:37.054211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b19d65-fb25-47ec-94c0-678b260507f6 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.055600 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b19d65-fb25-47ec-94c0-678b260507f6 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.055931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b19d65-fb25-47ec-94c0-678b260507f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.071450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 897/1821] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:36 2022] PUT /identity/v3/projects/fdc613edd261411fa7c9352fd2d0285e/users/1fbaf5030dfa409bb52b7d0b737bdb99/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:37.072905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a4644-add3-449c-9bdc-55d277c82d9e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:37.094852 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-25b19d65-fb25-47ec-94c0-678b260507f6 None None] MFA Rules not processed for user `6938cfb82a8e4a458b2e62a520f16eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:37.096106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-25b19d65-fb25-47ec-94c0-678b260507f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:37.150331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a4644-add3-449c-9bdc-55d277c82d9e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:37.155863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1257323-9527-4947-9aa7-03c489e84ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:37.156607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1257323-9527-4947-9aa7-03c489e84ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.156886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1257323-9527-4947-9aa7-03c489e84ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.157163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1257323-9527-4947-9aa7-03c489e84ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.158096 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1257323-9527-4947-9aa7-03c489e84ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:37.160233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1257323-9527-4947-9aa7-03c489e84ab9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:37.174805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 898/1822] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:37.174805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1930c-3672-491c-8245-cd8252394552 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.174805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1930c-3672-491c-8245-cd8252394552 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.175167 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1930c-3672-491c-8245-cd8252394552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.178130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 925/1823] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:37 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:37.178955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b19d65-fb25-47ec-94c0-678b260507f6 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:37.196875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b19d65-fb25-47ec-94c0-678b260507f6 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:37.204309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178c7a8-218c-432e-93a3-95aac33707b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:37.205006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178c7a8-218c-432e-93a3-95aac33707b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.205271 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178c7a8-218c-432e-93a3-95aac33707b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.205545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178c7a8-218c-432e-93a3-95aac33707b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.206535 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178c7a8-218c-432e-93a3-95aac33707b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:37.208249 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178c7a8-218c-432e-93a3-95aac33707b9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:37.209468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7cb1930c-3672-491c-8245-cd8252394552 None None] MFA Rules not processed for user `1fbaf5030dfa409bb52b7d0b737bdb99`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:37.210866 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7cb1930c-3672-491c-8245-cd8252394552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:37.220423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 926/1824] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:37.228528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178c7a8-218c-432e-93a3-95aac33707b9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:37.233093 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:37.241921 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178c7a8-218c-432e-93a3-95aac33707b9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:37.246788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898c1e4-1269-4721-b681-38215dd6c2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:37.247711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898c1e4-1269-4721-b681-38215dd6c2ca admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.247995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898c1e4-1269-4721-b681-38215dd6c2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.248300 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898c1e4-1269-4721-b681-38215dd6c2ca admin admin] PATH_INFO: `/v3/projects/5cd96ec8d7f24d8e9ad496ff21fa8824/users/01237d98a25a4b66a6eb8a03df1f2b70/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.270087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898c1e4-1269-4721-b681-38215dd6c2ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd96ec8d7f24d8e9ad496ff21fa8824, user_id=01237d98a25a4b66a6eb8a03df1f2b70, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:37.273306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898c1e4-1269-4721-b681-38215dd6c2ca admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:37.289933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 927/1825] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:37 2022] PUT /identity/v3/projects/5cd96ec8d7f24d8e9ad496ff21fa8824/users/01237d98a25a4b66a6eb8a03df1f2b70/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:37.299710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef8f8c-8fad-4d87-8199-cca25febce43 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.300139 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef8f8c-8fad-4d87-8199-cca25febce43 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.300329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef8f8c-8fad-4d87-8199-cca25febce43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.303180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 899/1826] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:37.308574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1930c-3672-491c-8245-cd8252394552 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:37.319525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1930c-3672-491c-8245-cd8252394552 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:37.336207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-06ef8f8c-8fad-4d87-8199-cca25febce43 None None] MFA Rules not processed for user `01237d98a25a4b66a6eb8a03df1f2b70`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:37.337554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-06ef8f8c-8fad-4d87-8199-cca25febce43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:37.381034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f6202-7d87-4155-bc64-bd9638d952e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:37.381977 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f6202-7d87-4155-bc64-bd9638d952e1 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.381977 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f6202-7d87-4155-bc64-bd9638d952e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.382250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f6202-7d87-4155-bc64-bd9638d952e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.382948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f6202-7d87-4155-bc64-bd9638d952e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:37.384926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f6202-7d87-4155-bc64-bd9638d952e1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:37.420744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 900/1827] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:37 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:37.425779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 928/1828] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:37.425958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f6202-7d87-4155-bc64-bd9638d952e1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:37.426559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef8f8c-8fad-4d87-8199-cca25febce43 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:37.438564 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef8f8c-8fad-4d87-8199-cca25febce43 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:37.515146 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f6202-7d87-4155-bc64-bd9638d952e1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:37.522034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c3ffd-d237-4462-b614-52ca125b503c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:37.523868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c3ffd-d237-4462-b614-52ca125b503c admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.524308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c3ffd-d237-4462-b614-52ca125b503c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.524588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c3ffd-d237-4462-b614-52ca125b503c admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.525304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c3ffd-d237-4462-b614-52ca125b503c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:37.527547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c3ffd-d237-4462-b614-52ca125b503c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:37.528338 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:37.571532 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 901/1829] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:37 2022] POST /identity/v3/users => generated 298 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:37.576045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c3ffd-d237-4462-b614-52ca125b503c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:37.592471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c3ffd-d237-4462-b614-52ca125b503c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:37.593022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1583bcd-d91d-4603-ba10-3352fe1d53c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:37.593963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1583bcd-d91d-4603-ba10-3352fe1d53c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.594488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1583bcd-d91d-4603-ba10-3352fe1d53c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.594804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1583bcd-d91d-4603-ba10-3352fe1d53c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.595589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1583bcd-d91d-4603-ba10-3352fe1d53c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:37.597652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1583bcd-d91d-4603-ba10-3352fe1d53c7 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:37.611042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77940c2-0461-433f-b6a0-afdd00610420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:37.613108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77940c2-0461-433f-b6a0-afdd00610420 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.614704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77940c2-0461-433f-b6a0-afdd00610420 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.615411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77940c2-0461-433f-b6a0-afdd00610420 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.617980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77940c2-0461-433f-b6a0-afdd00610420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:37.627008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77940c2-0461-433f-b6a0-afdd00610420 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:37.648673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 902/1830] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:49:37.648673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77940c2-0461-433f-b6a0-afdd00610420 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:37.655400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 929/1831] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:37 2022] POST /identity/v3/projects => generated 387 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:37.658321 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77940c2-0461-433f-b6a0-afdd00610420 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:37.659465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1583bcd-d91d-4603-ba10-3352fe1d53c7 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:37.691081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:37.793120 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d72f8-b44e-4e82-baa6-4b1f9ec7974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:37.793681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d72f8-b44e-4e82-baa6-4b1f9ec7974c admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.793963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d72f8-b44e-4e82-baa6-4b1f9ec7974c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.794269 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d72f8-b44e-4e82-baa6-4b1f9ec7974c admin admin] PATH_INFO: `/v3/projects/fd2acb1ca0654938996b0b2722bf6153/users/cf436cab698945c58a52595c5201b114/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.822347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1583bcd-d91d-4603-ba10-3352fe1d53c7 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:37.838070 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded638a-665f-4bfb-a811-6de25d8b5207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:37.838784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded638a-665f-4bfb-a811-6de25d8b5207 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.839115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded638a-665f-4bfb-a811-6de25d8b5207 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.840943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d72f8-b44e-4e82-baa6-4b1f9ec7974c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd2acb1ca0654938996b0b2722bf6153, user_id=cf436cab698945c58a52595c5201b114, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:37.848059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded638a-665f-4bfb-a811-6de25d8b5207 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.849949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded638a-665f-4bfb-a811-6de25d8b5207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:37.851959 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded638a-665f-4bfb-a811-6de25d8b5207 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:37.855315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d72f8-b44e-4e82-baa6-4b1f9ec7974c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:37.880782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 903/1832] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:37 2022] PUT /identity/v3/projects/fd2acb1ca0654938996b0b2722bf6153/users/cf436cab698945c58a52595c5201b114/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:37.881725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d72f8-b44e-4e82-baa6-4b1f9ec7974c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:37.893563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 930/1833] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:37 2022] POST /identity/v3/users => generated 300 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:37.894332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded638a-665f-4bfb-a811-6de25d8b5207 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:37.911464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded638a-665f-4bfb-a811-6de25d8b5207 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:37.981472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d72f8-b44e-4e82-baa6-4b1f9ec7974c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:37.993375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbffb2-333a-4860-9b87-a3ceaeab9455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:37.994187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbffb2-333a-4860-9b87-a3ceaeab9455 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:37.994552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbffb2-333a-4860-9b87-a3ceaeab9455 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:37.994901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbffb2-333a-4860-9b87-a3ceaeab9455 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:37.995714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbffb2-333a-4860-9b87-a3ceaeab9455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:37.997464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbffb2-333a-4860-9b87-a3ceaeab9455 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:38.019372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 904/1834] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:38.024150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbffb2-333a-4860-9b87-a3ceaeab9455 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:38.041329 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbffb2-333a-4860-9b87-a3ceaeab9455 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:38.047623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e42fac-7b19-433b-b579-d70ea1da7187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:38.048871 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e42fac-7b19-433b-b579-d70ea1da7187 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:38.049160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e42fac-7b19-433b-b579-d70ea1da7187 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:38.049522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e42fac-7b19-433b-b579-d70ea1da7187 admin admin] PATH_INFO: `/v3/projects/fd2acb1ca0654938996b0b2722bf6153/users/cf436cab698945c58a52595c5201b114/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:38.051593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e42fac-7b19-433b-b579-d70ea1da7187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd2acb1ca0654938996b0b2722bf6153, user_id=cf436cab698945c58a52595c5201b114, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:38.054614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e42fac-7b19-433b-b579-d70ea1da7187 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:38.056782 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381aab5-f7cb-4523-a20a-86c070a25227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:38.058163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381aab5-f7cb-4523-a20a-86c070a25227 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:38.058671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381aab5-f7cb-4523-a20a-86c070a25227 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:38.059050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381aab5-f7cb-4523-a20a-86c070a25227 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:38.059894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381aab5-f7cb-4523-a20a-86c070a25227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:38.068628 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 905/1835] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:38 2022] PUT /identity/v3/projects/fd2acb1ca0654938996b0b2722bf6153/users/cf436cab698945c58a52595c5201b114/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:38.069964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381aab5-f7cb-4523-a20a-86c070a25227 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:38.084177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf1cdd-2d95-4449-abae-4f62f3ed0e3e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:38.084177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf1cdd-2d95-4449-abae-4f62f3ed0e3e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:38.084177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf1cdd-2d95-4449-abae-4f62f3ed0e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:38.088955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 931/1836] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:38.097331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381aab5-f7cb-4523-a20a-86c070a25227 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:38.123974 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381aab5-f7cb-4523-a20a-86c070a25227 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:38.140358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8edf1cdd-2d95-4449-abae-4f62f3ed0e3e None None] MFA Rules not processed for user `cf436cab698945c58a52595c5201b114`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:38.141824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8edf1cdd-2d95-4449-abae-4f62f3ed0e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:38.237903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:38.258910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 906/1837] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:38.261562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d770a7e-f2b6-4c58-8f3a-f7eea5145f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:38.262502 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d770a7e-f2b6-4c58-8f3a-f7eea5145f52 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:38.262791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d770a7e-f2b6-4c58-8f3a-f7eea5145f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:38.263073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d770a7e-f2b6-4c58-8f3a-f7eea5145f52 admin admin] PATH_INFO: `/v3/projects/b33e540f31ec47d6a4c60bb9705a14e0/users/6fd0255457014b308826d7ab72475fef/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:38.272649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf1cdd-2d95-4449-abae-4f62f3ed0e3e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:38.274429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8edf1cdd-2d95-4449-abae-4f62f3ed0e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:38.282603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d770a7e-f2b6-4c58-8f3a-f7eea5145f52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33e540f31ec47d6a4c60bb9705a14e0, user_id=6fd0255457014b308826d7ab72475fef, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:38.285639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d770a7e-f2b6-4c58-8f3a-f7eea5145f52 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:38.317064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:38.321659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 932/1838] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:38 2022] PUT /identity/v3/projects/b33e540f31ec47d6a4c60bb9705a14e0/users/6fd0255457014b308826d7ab72475fef/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:38.326456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d770a7e-f2b6-4c58-8f3a-f7eea5145f52 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:38.421403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf1cdd-2d95-4449-abae-4f62f3ed0e3e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:38.523189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813704d-adce-4a05-9252-3ac80716c0fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:38.524656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813704d-adce-4a05-9252-3ac80716c0fc service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:38.525637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813704d-adce-4a05-9252-3ac80716c0fc service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:38.526460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813704d-adce-4a05-9252-3ac80716c0fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:38.560081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d770a7e-f2b6-4c58-8f3a-f7eea5145f52 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:38.565441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce8623-4610-4c89-b6b2-70cf9d2a80f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:38.566255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce8623-4610-4c89-b6b2-70cf9d2a80f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:38.566634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce8623-4610-4c89-b6b2-70cf9d2a80f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:38.567005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce8623-4610-4c89-b6b2-70cf9d2a80f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:38.567959 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce8623-4610-4c89-b6b2-70cf9d2a80f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:38.574620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce8623-4610-4c89-b6b2-70cf9d2a80f4 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:38.591597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 933/1839] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:38.595338 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658947d1-6596-4bde-8ef0-3439325056a9 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:38.595624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658947d1-6596-4bde-8ef0-3439325056a9 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:38.596041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658947d1-6596-4bde-8ef0-3439325056a9 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:38.603756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 934/1840] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:49:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:49:38.604713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658947d1-6596-4bde-8ef0-3439325056a9 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:38.638014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658947d1-6596-4bde-8ef0-3439325056a9 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:38.654884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c454fb-4916-4651-ab33-86603f1badc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:38.655970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c454fb-4916-4651-ab33-86603f1badc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:38.656284 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c454fb-4916-4651-ab33-86603f1badc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:38.656560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c454fb-4916-4651-ab33-86603f1badc4 admin admin] PATH_INFO: `/v3/projects/b33e540f31ec47d6a4c60bb9705a14e0/users/6fd0255457014b308826d7ab72475fef/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:38.661341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c454fb-4916-4651-ab33-86603f1badc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33e540f31ec47d6a4c60bb9705a14e0, user_id=6fd0255457014b308826d7ab72475fef, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:38.664876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c454fb-4916-4651-ab33-86603f1badc4 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:38.702669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 935/1841] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:38 2022] PUT /identity/v3/projects/b33e540f31ec47d6a4c60bb9705a14e0/users/6fd0255457014b308826d7ab72475fef/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:38.702669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd274a7-717e-4f24-abe1-94d71fc1baa4 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:38.702669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd274a7-717e-4f24-abe1-94d71fc1baa4 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:38.702669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd274a7-717e-4f24-abe1-94d71fc1baa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:38.703064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813704d-adce-4a05-9252-3ac80716c0fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:38.703064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813704d-adce-4a05-9252-3ac80716c0fc service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:38.729412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 907/1842] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:49:38.732635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dc91c-9860-4f6c-a520-7901bcc57676 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:38.732945 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dc91c-9860-4f6c-a520-7901bcc57676 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:38.733208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883dc91c-9860-4f6c-a520-7901bcc57676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:38.750116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-4fd274a7-717e-4f24-abe1-94d71fc1baa4 None None] MFA Rules not processed for user `eded06390a2744c9aa52f77e3bfc2db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:38.753869 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4fd274a7-717e-4f24-abe1-94d71fc1baa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:38.790187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-883dc91c-9860-4f6c-a520-7901bcc57676 None None] MFA Rules not processed for user `6fd0255457014b308826d7ab72475fef`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:38.791563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-883dc91c-9860-4f6c-a520-7901bcc57676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:38.919399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 936/1843] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:49:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:38.967121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd274a7-717e-4f24-abe1-94d71fc1baa4 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:38.969212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4fd274a7-717e-4f24-abe1-94d71fc1baa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:38.991472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd274a7-717e-4f24-abe1-94d71fc1baa4 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:39.004037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1513fad-55f6-40f6-b370-2991c8375c7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:39.005187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1513fad-55f6-40f6-b370-2991c8375c7a service placement] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:39.005600 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1513fad-55f6-40f6-b370-2991c8375c7a service placement] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:39.006024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1513fad-55f6-40f6-b370-2991c8375c7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:39.009834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 908/1844] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:39.022824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883dc91c-9860-4f6c-a520-7901bcc57676 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:39.026947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1513fad-55f6-40f6-b370-2991c8375c7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:39.029151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1513fad-55f6-40f6-b370-2991c8375c7a service placement] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:39.036061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:39.049884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 937/1845] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:39.173668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1513fad-55f6-40f6-b370-2991c8375c7a service placement] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:39.175850 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d1513fad-55f6-40f6-b370-2991c8375c7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:39.193641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883dc91c-9860-4f6c-a520-7901bcc57676 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:39.197712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1513fad-55f6-40f6-b370-2991c8375c7a service placement] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:39.197872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21437dcb-7ff1-4ff0-9ccb-a8792336efc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:39.198601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437dcb-7ff1-4ff0-9ccb-a8792336efc6 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:39.198882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437dcb-7ff1-4ff0-9ccb-a8792336efc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:39.199152 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437dcb-7ff1-4ff0-9ccb-a8792336efc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:39.212384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe82211-7fbd-46ea-ad4f-a7f1be49c615 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:39.213442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe82211-7fbd-46ea-ad4f-a7f1be49c615 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:39.214372 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe82211-7fbd-46ea-ad4f-a7f1be49c615 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:39.215698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe82211-7fbd-46ea-ad4f-a7f1be49c615 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:39.243821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe82211-7fbd-46ea-ad4f-a7f1be49c615 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:39.245553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe82211-7fbd-46ea-ad4f-a7f1be49c615 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:39.280075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 938/1846] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:49:39.304977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe82211-7fbd-46ea-ad4f-a7f1be49c615 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:39.337861 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21437dcb-7ff1-4ff0-9ccb-a8792336efc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:39.339017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21437dcb-7ff1-4ff0-9ccb-a8792336efc6 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:39.361058 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 909/1847] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:39.370656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:39.462900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe82211-7fbd-46ea-ad4f-a7f1be49c615 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:39.468532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:39.469837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:39.470132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:39.470585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] PATH_INFO: `/v3/users/01237d98a25a4b66a6eb8a03df1f2b70` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:39.472567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01237d98a25a4b66a6eb8a03df1f2b70)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:39.475049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:39.577927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01237d98a25a4b66a6eb8a03df1f2b70'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:39.578710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01237d98a25a4b66a6eb8a03df1f2b70'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:39.594737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01237d98a25a4b66a6eb8a03df1f2b70'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:39.606059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01237d98a25a4b66a6eb8a03df1f2b70'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:39.636951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01237d98a25a4b66a6eb8a03df1f2b70'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:39.655415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 939/1848] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:39 2022] DELETE /identity/v3/users/01237d98a25a4b66a6eb8a03df1f2b70 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:39.663222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06616808-cd05-4d99-b64d-9dec7a8a2e93 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:39.663222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06616808-cd05-4d99-b64d-9dec7a8a2e93 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:39.663222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06616808-cd05-4d99-b64d-9dec7a8a2e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:39.699500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-06616808-cd05-4d99-b64d-9dec7a8a2e93 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:39.700826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-06616808-cd05-4d99-b64d-9dec7a8a2e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:39.798325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 910/1849] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:39.825465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:39.828116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:40.034234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d519cae-77ca-4dd8-af28-f4be4a409724 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:40.045699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405e01f-51c9-49d0-856e-23913138432b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:40.046442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405e01f-51c9-49d0-856e-23913138432b service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:40.046737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405e01f-51c9-49d0-856e-23913138432b service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:40.047020 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405e01f-51c9-49d0-856e-23913138432b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:40.075231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405e01f-51c9-49d0-856e-23913138432b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:40.076909 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405e01f-51c9-49d0-856e-23913138432b service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:40.108366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 940/1850] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:40.257667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f633e-6a5e-498b-92f0-1794c48427a9 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:40.257917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f633e-6a5e-498b-92f0-1794c48427a9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:40.258173 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f633e-6a5e-498b-92f0-1794c48427a9 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:40.259578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 911/1851] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:49:40.277381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f46fd7-449f-45a9-93d0-7fcd3390dd6d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:40.279049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f46fd7-449f-45a9-93d0-7fcd3390dd6d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:40.279542 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f46fd7-449f-45a9-93d0-7fcd3390dd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:40.330684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-64f46fd7-449f-45a9-93d0-7fcd3390dd6d None None] MFA Rules not processed for user `eded06390a2744c9aa52f77e3bfc2db9`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:40.335190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-64f46fd7-449f-45a9-93d0-7fcd3390dd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:40.477939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:40.537854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 941/1852] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:49:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:40.538165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f633e-6a5e-498b-92f0-1794c48427a9 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:40.539885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f60f633e-6a5e-498b-92f0-1794c48427a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:40.590526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f46fd7-449f-45a9-93d0-7fcd3390dd6d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:40.592111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-64f46fd7-449f-45a9-93d0-7fcd3390dd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:40.593969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f633e-6a5e-498b-92f0-1794c48427a9 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:40.608128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:40.615251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:40.616074 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f46fd7-449f-45a9-93d0-7fcd3390dd6d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:40.617634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:40.618089 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] PATH_INFO: `/v3/users/dba7c0a9983946089fd66c1f3b12f327` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:40.623351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dba7c0a9983946089fd66c1f3b12f327)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:40.629641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bade49-955b-47ae-9675-fe1df340828f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:40.630223 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:40.631604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bade49-955b-47ae-9675-fe1df340828f service placement] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:40.632040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bade49-955b-47ae-9675-fe1df340828f service placement] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:40.633698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bade49-955b-47ae-9675-fe1df340828f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:40.658810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bade49-955b-47ae-9675-fe1df340828f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:40.660484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bade49-955b-47ae-9675-fe1df340828f service placement] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:40.683346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 942/1853] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:49:40.746727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dba7c0a9983946089fd66c1f3b12f327'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:40.782075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dba7c0a9983946089fd66c1f3b12f327'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:40.783379 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dba7c0a9983946089fd66c1f3b12f327'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:40.785429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:40.792971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dba7c0a9983946089fd66c1f3b12f327'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:40.814815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dba7c0a9983946089fd66c1f3b12f327'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:40.822991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 912/1854] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:40 2022] DELETE /identity/v3/users/dba7c0a9983946089fd66c1f3b12f327 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:40.884101 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bade49-955b-47ae-9675-fe1df340828f service placement] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:40.885604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-52bade49-955b-47ae-9675-fe1df340828f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:40.901791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bade49-955b-47ae-9675-fe1df340828f service placement] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:40.975610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:40.977096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:40.977389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:40.978098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] PATH_INFO: `/v3/projects/5cd96ec8d7f24d8e9ad496ff21fa8824` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:40.980102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cd96ec8d7f24d8e9ad496ff21fa8824)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:40.981753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:41.031900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cd96ec8d7f24d8e9ad496ff21fa8824', 'request_id': 'req-4e578578-7079-4de3-aea9-489152378161'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.041095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cd96ec8d7f24d8e9ad496ff21fa8824', 'request_id': 'req-4e578578-7079-4de3-aea9-489152378161'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.054084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cd96ec8d7f24d8e9ad496ff21fa8824', 'request_id': 'req-4e578578-7079-4de3-aea9-489152378161'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.067604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] The token cache is being invalidate because project 5cd96ec8d7f24d8e9ad496ff21fa8824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:41.067944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.068936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.070617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 943/1855] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:40 2022] DELETE /identity/v3/projects/5cd96ec8d7f24d8e9ad496ff21fa8824 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:41.074791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:41.076295 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:41.080678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:41.082102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:41.098186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae2ce8-85e5-471c-a441-66264435936e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:41.170836 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e578578-7079-4de3-aea9-489152378161 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:41.175727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:41.176703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:41.176989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:41.177267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] PATH_INFO: `/v3/projects/5683619f4ab7459bb5cc12d91703b28d` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:41.178659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5683619f4ab7459bb5cc12d91703b28d)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:41.180694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:41.183067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:41.192931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:41.193996 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:41.194240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:41.194738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] PATH_INFO: `/v3/users/6fd0255457014b308826d7ab72475fef` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:41.196178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fd0255457014b308826d7ab72475fef)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:41.198175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:41.235399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5683619f4ab7459bb5cc12d91703b28d', 'request_id': 'req-37ad8008-88ec-4fd4-936f-16e0963ca10a'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.244206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5683619f4ab7459bb5cc12d91703b28d', 'request_id': 'req-37ad8008-88ec-4fd4-936f-16e0963ca10a'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.248321 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fd0255457014b308826d7ab72475fef'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.251002 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5683619f4ab7459bb5cc12d91703b28d', 'request_id': 'req-37ad8008-88ec-4fd4-936f-16e0963ca10a'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.256573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fd0255457014b308826d7ab72475fef'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.257239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fd0255457014b308826d7ab72475fef'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.264395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] The token cache is being invalidate because project 5683619f4ab7459bb5cc12d91703b28d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:41.264745 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.264834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fd0255457014b308826d7ab72475fef'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.265359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.267041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 944/1856] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:41 2022] DELETE /identity/v3/projects/5683619f4ab7459bb5cc12d91703b28d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:41.271470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:41.272841 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:41.277578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fd0255457014b308826d7ab72475fef'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.283865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 913/1857] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:41 2022] DELETE /identity/v3/users/6fd0255457014b308826d7ab72475fef => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:41.343548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad8008-88ec-4fd4-936f-16e0963ca10a admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:41.404675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:41.405570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:41.405862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:41.406142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] PATH_INFO: `/v3/users/6938cfb82a8e4a458b2e62a520f16eb4` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:41.407366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6938cfb82a8e4a458b2e62a520f16eb4)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:41.409403 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:41.461342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:41.467697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6938cfb82a8e4a458b2e62a520f16eb4'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.468968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6938cfb82a8e4a458b2e62a520f16eb4'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.478041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6938cfb82a8e4a458b2e62a520f16eb4'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.485302 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6938cfb82a8e4a458b2e62a520f16eb4'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.497209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6938cfb82a8e4a458b2e62a520f16eb4'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.503318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 945/1858] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:41 2022] DELETE /identity/v3/users/6938cfb82a8e4a458b2e62a520f16eb4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:41.577677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:41.579748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:41.608245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8edd2d-9788-48fd-aa8a-843519f1ea25 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:41.661545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:41.681040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869dc8c4-4aab-495e-ae80-1d94535b056a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:41.682556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869dc8c4-4aab-495e-ae80-1d94535b056a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:41.682932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869dc8c4-4aab-495e-ae80-1d94535b056a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:41.683633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869dc8c4-4aab-495e-ae80-1d94535b056a admin admin] PATH_INFO: `/v3/projects/b33e540f31ec47d6a4c60bb9705a14e0` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:41.685283 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869dc8c4-4aab-495e-ae80-1d94535b056a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33e540f31ec47d6a4c60bb9705a14e0)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:41.688344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869dc8c4-4aab-495e-ae80-1d94535b056a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:41.704941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:41.708031 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:41.728329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00535c-e2ec-4aa1-8d1a-6be459af3f32 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:41.757226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:41.758135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:41.758422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:41.758720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9 admin admin] PATH_INFO: `/v3/projects/37203b7ae9cd41ebbb14b6852c817d38` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:41.760133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37203b7ae9cd41ebbb14b6852c817d38)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:41.762140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:41.776532 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-869dc8c4-4aab-495e-ae80-1d94535b056a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33e540f31ec47d6a4c60bb9705a14e0', 'request_id': 'req-869dc8c4-4aab-495e-ae80-1d94535b056a'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.788829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-869dc8c4-4aab-495e-ae80-1d94535b056a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33e540f31ec47d6a4c60bb9705a14e0', 'request_id': 'req-869dc8c4-4aab-495e-ae80-1d94535b056a'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.796869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-869dc8c4-4aab-495e-ae80-1d94535b056a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33e540f31ec47d6a4c60bb9705a14e0', 'request_id': 'req-869dc8c4-4aab-495e-ae80-1d94535b056a'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.810248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-869dc8c4-4aab-495e-ae80-1d94535b056a admin admin] The token cache is being invalidate because project b33e540f31ec47d6a4c60bb9705a14e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:41.810571 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-869dc8c4-4aab-495e-ae80-1d94535b056a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.811720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-869dc8c4-4aab-495e-ae80-1d94535b056a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.813859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 914/1859] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:41 2022] DELETE /identity/v3/projects/b33e540f31ec47d6a4c60bb9705a14e0 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:41.822091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1688b-40c9-4bd4-900c-113a15706d02 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:41.822697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1688b-40c9-4bd4-900c-113a15706d02 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:41.823305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1688b-40c9-4bd4-900c-113a15706d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:41.826860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37203b7ae9cd41ebbb14b6852c817d38', 'request_id': 'req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.835922 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37203b7ae9cd41ebbb14b6852c817d38', 'request_id': 'req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.848536 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37203b7ae9cd41ebbb14b6852c817d38', 'request_id': 'req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.849504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-0dc1688b-40c9-4bd4-900c-113a15706d02 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:41.851450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-0dc1688b-40c9-4bd4-900c-113a15706d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:41.875293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9 admin admin] The token cache is being invalidate because project 37203b7ae9cd41ebbb14b6852c817d38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:41.877715 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.879842 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-eef4e8c6-7cae-4fb2-acbe-8921b340b0a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:41.883578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 946/1860] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:41 2022] DELETE /identity/v3/projects/37203b7ae9cd41ebbb14b6852c817d38 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:41.895340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf41a5-dded-4ad9-b006-73356fd1bb03 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:41.895340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf41a5-dded-4ad9-b006-73356fd1bb03 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:41.895340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf41a5-dded-4ad9-b006-73356fd1bb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:41.931635 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-5ebf41a5-dded-4ad9-b006-73356fd1bb03 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:41.932889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5ebf41a5-dded-4ad9-b006-73356fd1bb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:41.986164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 915/1861] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:41.988998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a923f-7513-465e-8378-5b1dff82e8f8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:41.989376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a923f-7513-465e-8378-5b1dff82e8f8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:41.989722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a923f-7513-465e-8378-5b1dff82e8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.019750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-777a923f-7513-465e-8378-5b1dff82e8f8 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:42.021383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-777a923f-7513-465e-8378-5b1dff82e8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:42.024943 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 947/1862] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:42.032913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 916/1863] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:42.033201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd1ad4-7531-4657-bd7e-a2db0ab4a54b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.033500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd1ad4-7531-4657-bd7e-a2db0ab4a54b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.033832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd1ad4-7531-4657-bd7e-a2db0ab4a54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.036915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a923f-7513-465e-8378-5b1dff82e8f8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.046707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a923f-7513-465e-8378-5b1dff82e8f8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.055114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5916a-7a4d-4c2a-b14b-af21b673b0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.057953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5916a-7a4d-4c2a-b14b-af21b673b0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.058154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-24bd1ad4-7531-4657-bd7e-a2db0ab4a54b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:42.058308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5916a-7a4d-4c2a-b14b-af21b673b0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.058308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5916a-7a4d-4c2a-b14b-af21b673b0a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.058308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5916a-7a4d-4c2a-b14b-af21b673b0a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:42.058566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-24bd1ad4-7531-4657-bd7e-a2db0ab4a54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:42.058641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5916a-7a4d-4c2a-b14b-af21b673b0a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.059309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5916a-7a4d-4c2a-b14b-af21b673b0a9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.065669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 948/1864] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:42.069952 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd1ad4-7531-4657-bd7e-a2db0ab4a54b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.074262 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 917/1865] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:49:42.078253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5916a-7a4d-4c2a-b14b-af21b673b0a9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.079222 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd1ad4-7531-4657-bd7e-a2db0ab4a54b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.084482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f05323-ceb1-4f78-bf0c-05ab6f486433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.085067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f05323-ceb1-4f78-bf0c-05ab6f486433 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.085359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f05323-ceb1-4f78-bf0c-05ab6f486433 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.085625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f05323-ceb1-4f78-bf0c-05ab6f486433 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.086358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f05323-ceb1-4f78-bf0c-05ab6f486433 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:42.086667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f05323-ceb1-4f78-bf0c-05ab6f486433 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.088325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f05323-ceb1-4f78-bf0c-05ab6f486433 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.089561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5916a-7a4d-4c2a-b14b-af21b673b0a9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.094466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8e582-d01b-42fd-9135-23c0d9a1f404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.095137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8e582-d01b-42fd-9135-23c0d9a1f404 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.095410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8e582-d01b-42fd-9135-23c0d9a1f404 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.095685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8e582-d01b-42fd-9135-23c0d9a1f404 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.096429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8e582-d01b-42fd-9135-23c0d9a1f404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.098830 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8e582-d01b-42fd-9135-23c0d9a1f404 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.100931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 949/1866] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:49:42.105453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f05323-ceb1-4f78-bf0c-05ab6f486433 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.116096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f05323-ceb1-4f78-bf0c-05ab6f486433 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.121785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a5d15-ef53-499b-80eb-de81b74560a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.122459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a5d15-ef53-499b-80eb-de81b74560a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.122735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a5d15-ef53-499b-80eb-de81b74560a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.123013 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a5d15-ef53-499b-80eb-de81b74560a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.123808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a5d15-ef53-499b-80eb-de81b74560a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.125770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a5d15-ef53-499b-80eb-de81b74560a7 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.130237 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 918/1867] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:42 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:42.131070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8e582-d01b-42fd-9135-23c0d9a1f404 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.152505 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:42.162439 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 950/1868] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:42 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:42.166395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a5d15-ef53-499b-80eb-de81b74560a7 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.181504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:42.220899 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8e582-d01b-42fd-9135-23c0d9a1f404 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.262320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a5d15-ef53-499b-80eb-de81b74560a7 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.267682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c5ce8-7f72-43a1-a9a0-0cd9b1c52b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.268628 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c5ce8-7f72-43a1-a9a0-0cd9b1c52b1c admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.268915 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c5ce8-7f72-43a1-a9a0-0cd9b1c52b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.269286 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c5ce8-7f72-43a1-a9a0-0cd9b1c52b1c admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.270109 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c5ce8-7f72-43a1-a9a0-0cd9b1c52b1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.274491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c5ce8-7f72-43a1-a9a0-0cd9b1c52b1c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.303957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 951/1869] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:42 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:42.308597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c5ce8-7f72-43a1-a9a0-0cd9b1c52b1c admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.312381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6e72e-7fec-4889-862a-87bd99e8a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.313216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6e72e-7fec-4889-862a-87bd99e8a793 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.313513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6e72e-7fec-4889-862a-87bd99e8a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.313834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6e72e-7fec-4889-862a-87bd99e8a793 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.314593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6e72e-7fec-4889-862a-87bd99e8a793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.316597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6e72e-7fec-4889-862a-87bd99e8a793 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.320753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c5ce8-7f72-43a1-a9a0-0cd9b1c52b1c admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.326670 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79612546-2f30-4ab5-891d-79a218e157fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.327464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79612546-2f30-4ab5-891d-79a218e157fa admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.327913 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79612546-2f30-4ab5-891d-79a218e157fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.328227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79612546-2f30-4ab5-891d-79a218e157fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.329097 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79612546-2f30-4ab5-891d-79a218e157fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.330772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79612546-2f30-4ab5-891d-79a218e157fa admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.341664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 952/1870] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:42.342936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 919/1871] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:42 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:42.345705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79612546-2f30-4ab5-891d-79a218e157fa admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.346913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6e72e-7fec-4889-862a-87bd99e8a793 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.357391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6e72e-7fec-4889-862a-87bd99e8a793 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.357540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79612546-2f30-4ab5-891d-79a218e157fa admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.362711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0a967-a049-43ef-a06f-0e3599529424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.363393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0a967-a049-43ef-a06f-0e3599529424 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.363567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e26c0-ff02-44ec-affb-64dd201231ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.363711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0a967-a049-43ef-a06f-0e3599529424 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.363979 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb0a967-a049-43ef-a06f-0e3599529424 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.364304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e26c0-ff02-44ec-affb-64dd201231ab admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.364589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e26c0-ff02-44ec-affb-64dd201231ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.364855 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e26c0-ff02-44ec-affb-64dd201231ab admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.364934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0a967-a049-43ef-a06f-0e3599529424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.368152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e26c0-ff02-44ec-affb-64dd201231ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.368231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb0a967-a049-43ef-a06f-0e3599529424 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.368335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e26c0-ff02-44ec-affb-64dd201231ab admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.378805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 920/1872] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:42.381711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 953/1873] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:49:42.386360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0a967-a049-43ef-a06f-0e3599529424 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.386792 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e26c0-ff02-44ec-affb-64dd201231ab admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.397762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb0a967-a049-43ef-a06f-0e3599529424 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.403397 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682da5da-41d8-4d98-a850-d948662cef6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.404276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682da5da-41d8-4d98-a850-d948662cef6c admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.404611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682da5da-41d8-4d98-a850-d948662cef6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.404969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682da5da-41d8-4d98-a850-d948662cef6c admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.405071 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e26c0-ff02-44ec-affb-64dd201231ab admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.405768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682da5da-41d8-4d98-a850-d948662cef6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.407436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682da5da-41d8-4d98-a850-d948662cef6c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.410331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db37f04-7ddc-41d0-a41b-4106b620d448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.411323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db37f04-7ddc-41d0-a41b-4106b620d448 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.411603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db37f04-7ddc-41d0-a41b-4106b620d448 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.411881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db37f04-7ddc-41d0-a41b-4106b620d448 admin admin] PATH_INFO: `/v3/projects/d849a86eb6f44b2cb5b45bef5170b71a/users/842320d334c143719abd60d48bb53d2f/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.420110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 921/1874] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:42.423982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682da5da-41d8-4d98-a850-d948662cef6c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.429392 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db37f04-7ddc-41d0-a41b-4106b620d448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d849a86eb6f44b2cb5b45bef5170b71a, user_id=842320d334c143719abd60d48bb53d2f, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.432251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db37f04-7ddc-41d0-a41b-4106b620d448 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.435951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682da5da-41d8-4d98-a850-d948662cef6c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.441486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd7b4c-ea15-4af4-9f8d-40983ee7d9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.442752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd7b4c-ea15-4af4-9f8d-40983ee7d9c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.443036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd7b4c-ea15-4af4-9f8d-40983ee7d9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.443314 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd7b4c-ea15-4af4-9f8d-40983ee7d9c0 admin admin] PATH_INFO: `/v3/projects/dad61a7d4edd4c1cb22ef642af3acab6/users/75647c03a37944979657ac3ecd6cc5d2/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.452204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 954/1875] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:42 2022] PUT /identity/v3/projects/d849a86eb6f44b2cb5b45bef5170b71a/users/842320d334c143719abd60d48bb53d2f/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:42.465428 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a86e0-e5ec-4b79-a4aa-cc6acf3ba3cb None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.465920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a86e0-e5ec-4b79-a4aa-cc6acf3ba3cb None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.466374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a86e0-e5ec-4b79-a4aa-cc6acf3ba3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.471742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd7b4c-ea15-4af4-9f8d-40983ee7d9c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad61a7d4edd4c1cb22ef642af3acab6, user_id=75647c03a37944979657ac3ecd6cc5d2, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.476971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd7b4c-ea15-4af4-9f8d-40983ee7d9c0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.496324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 922/1876] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:42 2022] PUT /identity/v3/projects/dad61a7d4edd4c1cb22ef642af3acab6/users/75647c03a37944979657ac3ecd6cc5d2/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:42.503874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-797a86e0-e5ec-4b79-a4aa-cc6acf3ba3cb None None] MFA Rules not processed for user `842320d334c143719abd60d48bb53d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:42.505199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-797a86e0-e5ec-4b79-a4aa-cc6acf3ba3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:42.509717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9045c0-69af-43eb-9b4a-b86f1ecb4071 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.510051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9045c0-69af-43eb-9b4a-b86f1ecb4071 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.510374 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9045c0-69af-43eb-9b4a-b86f1ecb4071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.538164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-fd9045c0-69af-43eb-9b4a-b86f1ecb4071 None None] MFA Rules not processed for user `75647c03a37944979657ac3ecd6cc5d2`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:42.539662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fd9045c0-69af-43eb-9b4a-b86f1ecb4071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:42.578056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 955/1877] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:42.582532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a86e0-e5ec-4b79-a4aa-cc6acf3ba3cb None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.602562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 923/1878] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:42.606972 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9045c0-69af-43eb-9b4a-b86f1ecb4071 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.616375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:42.627921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:42.649596 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a86e0-e5ec-4b79-a4aa-cc6acf3ba3cb None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.654663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f4093-0af4-4512-8860-0ae0fe803a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.655330 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f4093-0af4-4512-8860-0ae0fe803a9c admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.655890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f4093-0af4-4512-8860-0ae0fe803a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.656197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f4093-0af4-4512-8860-0ae0fe803a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.656894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f4093-0af4-4512-8860-0ae0fe803a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.658881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f4093-0af4-4512-8860-0ae0fe803a9c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.674551 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9045c0-69af-43eb-9b4a-b86f1ecb4071 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.680113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c15727-14e6-4fc9-bd43-9cd553cecbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.680785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c15727-14e6-4fc9-bd43-9cd553cecbfc admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.681068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c15727-14e6-4fc9-bd43-9cd553cecbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.681343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c15727-14e6-4fc9-bd43-9cd553cecbfc admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.682039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c15727-14e6-4fc9-bd43-9cd553cecbfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.684334 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c15727-14e6-4fc9-bd43-9cd553cecbfc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.685431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 956/1879] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:42 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:42.689036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f4093-0af4-4512-8860-0ae0fe803a9c admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.711139 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 924/1880] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:42 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:42.715202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c15727-14e6-4fc9-bd43-9cd553cecbfc admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.753007 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f4093-0af4-4512-8860-0ae0fe803a9c admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.786425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c15727-14e6-4fc9-bd43-9cd553cecbfc admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.791374 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1f12a-a04c-4f36-a389-b553a2e0d91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.792089 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1f12a-a04c-4f36-a389-b553a2e0d91f admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.792382 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1f12a-a04c-4f36-a389-b553a2e0d91f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.792659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1f12a-a04c-4f36-a389-b553a2e0d91f admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.793360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1f12a-a04c-4f36-a389-b553a2e0d91f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.795378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1f12a-a04c-4f36-a389-b553a2e0d91f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.821145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 925/1881] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:42 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:42.823560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829cc25-b4f4-440d-81c7-abf2c7472ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.824389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829cc25-b4f4-440d-81c7-abf2c7472ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.824782 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829cc25-b4f4-440d-81c7-abf2c7472ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.825103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829cc25-b4f4-440d-81c7-abf2c7472ca8 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.825831 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829cc25-b4f4-440d-81c7-abf2c7472ca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.828137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9829cc25-b4f4-440d-81c7-abf2c7472ca8 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.831757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1f12a-a04c-4f36-a389-b553a2e0d91f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.842642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1f12a-a04c-4f36-a389-b553a2e0d91f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.847390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6efe4c-7393-41a9-90d6-663c7fd1b0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.848144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6efe4c-7393-41a9-90d6-663c7fd1b0fa admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.848433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6efe4c-7393-41a9-90d6-663c7fd1b0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.848710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6efe4c-7393-41a9-90d6-663c7fd1b0fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.849438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6efe4c-7393-41a9-90d6-663c7fd1b0fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.850173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 957/1882] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:42 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:42.851062 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6efe4c-7393-41a9-90d6-663c7fd1b0fa admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.854002 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829cc25-b4f4-440d-81c7-abf2c7472ca8 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.861404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 926/1883] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:49:42.864868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829cc25-b4f4-440d-81c7-abf2c7472ca8 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.865372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6efe4c-7393-41a9-90d6-663c7fd1b0fa admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.869544 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6878c-195c-4750-84da-826048d5e72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.870210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6878c-195c-4750-84da-826048d5e72e admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.870500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6878c-195c-4750-84da-826048d5e72e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.870778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6878c-195c-4750-84da-826048d5e72e admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.871529 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6878c-195c-4750-84da-826048d5e72e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.873183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6878c-195c-4750-84da-826048d5e72e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.874857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6efe4c-7393-41a9-90d6-663c7fd1b0fa admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.879442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc8d86-cd72-41bb-bfc8-2c879a6ab3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.880353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc8d86-cd72-41bb-bfc8-2c879a6ab3ec admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.880640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc8d86-cd72-41bb-bfc8-2c879a6ab3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.880923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc8d86-cd72-41bb-bfc8-2c879a6ab3ec admin admin] PATH_INFO: `/v3/projects/ae3410896d1747859c1ed4acddacad98/users/433f06f7a1ed414ea06abac1abdf0cee/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.887044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 958/1884] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:49:42.888057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6878c-195c-4750-84da-826048d5e72e admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.896486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc8d86-cd72-41bb-bfc8-2c879a6ab3ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3410896d1747859c1ed4acddacad98, user_id=433f06f7a1ed414ea06abac1abdf0cee, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.898023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6878c-195c-4750-84da-826048d5e72e admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.899197 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc8d86-cd72-41bb-bfc8-2c879a6ab3ec admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.903810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ca9b1-19ef-4f5c-83a0-210a202ffa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:42.904884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ca9b1-19ef-4f5c-83a0-210a202ffa20 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:42.905226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ca9b1-19ef-4f5c-83a0-210a202ffa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:42.905597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ca9b1-19ef-4f5c-83a0-210a202ffa20 admin admin] PATH_INFO: `/v3/projects/d6a4d674c9734956bc5bc3498d15e678/users/5f871bb778e14e0193dc5fa43c090f00/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:42.913475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 927/1885] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:42 2022] PUT /identity/v3/projects/ae3410896d1747859c1ed4acddacad98/users/433f06f7a1ed414ea06abac1abdf0cee/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:42.917372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc8d86-cd72-41bb-bfc8-2c879a6ab3ec admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.925118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ca9b1-19ef-4f5c-83a0-210a202ffa20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a4d674c9734956bc5bc3498d15e678, user_id=5f871bb778e14e0193dc5fa43c090f00, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:42.927873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ca9b1-19ef-4f5c-83a0-210a202ffa20 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:42.942205 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 959/1886] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:42 2022] PUT /identity/v3/projects/d6a4d674c9734956bc5bc3498d15e678/users/5f871bb778e14e0193dc5fa43c090f00/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:42.947508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ca9b1-19ef-4f5c-83a0-210a202ffa20 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:42.970396 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:42.987744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc8d86-cd72-41bb-bfc8-2c879a6ab3ec admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:42.991259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:43.041301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ca9b1-19ef-4f5c-83a0-210a202ffa20 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:43.050194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d484a58-b5e1-46cb-bac2-2fa1e585d4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:43.050896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d484a58-b5e1-46cb-bac2-2fa1e585d4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:43.051371 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d484a58-b5e1-46cb-bac2-2fa1e585d4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:43.051652 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d484a58-b5e1-46cb-bac2-2fa1e585d4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:43.052452 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d484a58-b5e1-46cb-bac2-2fa1e585d4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:43.054464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d484a58-b5e1-46cb-bac2-2fa1e585d4e7 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:43.066538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 960/1887] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:43.071060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d484a58-b5e1-46cb-bac2-2fa1e585d4e7 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:43.074426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdf60c-8eab-453a-9709-e6217962fc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:43.075126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdf60c-8eab-453a-9709-e6217962fc7b admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:43.075414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdf60c-8eab-453a-9709-e6217962fc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:43.075690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdf60c-8eab-453a-9709-e6217962fc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:43.076473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdf60c-8eab-453a-9709-e6217962fc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:43.079126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdf60c-8eab-453a-9709-e6217962fc7b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:43.083910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d484a58-b5e1-46cb-bac2-2fa1e585d4e7 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:43.091544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 928/1888] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:49:43.093428 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cf26f-2092-49ae-a04d-6fa73bc6320a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:43.094850 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cf26f-2092-49ae-a04d-6fa73bc6320a admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:43.095478 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cf26f-2092-49ae-a04d-6fa73bc6320a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:43.096127 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cf26f-2092-49ae-a04d-6fa73bc6320a admin admin] PATH_INFO: `/v3/projects/d6a4d674c9734956bc5bc3498d15e678/users/5f871bb778e14e0193dc5fa43c090f00/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:43.099291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cf26f-2092-49ae-a04d-6fa73bc6320a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a4d674c9734956bc5bc3498d15e678, user_id=5f871bb778e14e0193dc5fa43c090f00, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:43.104640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdf60c-8eab-453a-9709-e6217962fc7b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:43.110756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cf26f-2092-49ae-a04d-6fa73bc6320a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:43.114781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdf60c-8eab-453a-9709-e6217962fc7b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:43.119562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e0087-dd07-490a-a705-4d9763e3a9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:43.120569 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e0087-dd07-490a-a705-4d9763e3a9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:43.120919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e0087-dd07-490a-a705-4d9763e3a9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:43.121259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e0087-dd07-490a-a705-4d9763e3a9cd admin admin] PATH_INFO: `/v3/projects/ae3410896d1747859c1ed4acddacad98/users/433f06f7a1ed414ea06abac1abdf0cee/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:43.123401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 961/1889] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:43 2022] PUT /identity/v3/projects/d6a4d674c9734956bc5bc3498d15e678/users/5f871bb778e14e0193dc5fa43c090f00/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:43.123521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e0087-dd07-490a-a705-4d9763e3a9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3410896d1747859c1ed4acddacad98, user_id=433f06f7a1ed414ea06abac1abdf0cee, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:43.126145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e0087-dd07-490a-a705-4d9763e3a9cd admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:43.134183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fec4d-6302-41b3-81e8-25d53b52cfa4 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:43.134580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fec4d-6302-41b3-81e8-25d53b52cfa4 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:43.134928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fec4d-6302-41b3-81e8-25d53b52cfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:43.146487 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 929/1890] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:43 2022] PUT /identity/v3/projects/ae3410896d1747859c1ed4acddacad98/users/433f06f7a1ed414ea06abac1abdf0cee/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:43.157380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f6abe-3a08-477a-a1fa-00786f9d7fb3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:43.157670 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f6abe-3a08-477a-a1fa-00786f9d7fb3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:43.157946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f6abe-3a08-477a-a1fa-00786f9d7fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:43.163739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f65fec4d-6302-41b3-81e8-25d53b52cfa4 None None] MFA Rules not processed for user `5f871bb778e14e0193dc5fa43c090f00`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:43.165069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f65fec4d-6302-41b3-81e8-25d53b52cfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:43.186877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-fe7f6abe-3a08-477a-a1fa-00786f9d7fb3 None None] MFA Rules not processed for user `433f06f7a1ed414ea06abac1abdf0cee`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:43.187784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fe7f6abe-3a08-477a-a1fa-00786f9d7fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:43.246140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 962/1891] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:43.258363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fec4d-6302-41b3-81e8-25d53b52cfa4 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:43.259746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f65fec4d-6302-41b3-81e8-25d53b52cfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:43.261386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 930/1892] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:43.273082 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f6abe-3a08-477a-a1fa-00786f9d7fb3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:43.274583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fe7f6abe-3a08-477a-a1fa-00786f9d7fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:43.317152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fec4d-6302-41b3-81e8-25d53b52cfa4 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:43.321356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5dde-12b2-4bbb-9c80-56f827340903 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:43.322149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5dde-12b2-4bbb-9c80-56f827340903 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:43.322438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5dde-12b2-4bbb-9c80-56f827340903 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:43.322717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5dde-12b2-4bbb-9c80-56f827340903 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:43.330640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f6abe-3a08-477a-a1fa-00786f9d7fb3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:43.332794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb5dde-12b2-4bbb-9c80-56f827340903 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:43.334303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb5dde-12b2-4bbb-9c80-56f827340903 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:43.334775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d62ae3-225f-413e-9572-a17e1a6166f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:43.336827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d62ae3-225f-413e-9572-a17e1a6166f2 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:43.336827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d62ae3-225f-413e-9572-a17e1a6166f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:43.336827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d62ae3-225f-413e-9572-a17e1a6166f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:43.344902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 963/1893] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:49:43.352270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d62ae3-225f-413e-9572-a17e1a6166f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:43.353763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d62ae3-225f-413e-9572-a17e1a6166f2 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:43.364740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 931/1894] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:49:43.429510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5dde-12b2-4bbb-9c80-56f827340903 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:43.439037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5dde-12b2-4bbb-9c80-56f827340903 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:43.443227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de90498-b1c3-4a16-b515-436ad4e8e681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:43.443955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de90498-b1c3-4a16-b515-436ad4e8e681 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:43.444429 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de90498-b1c3-4a16-b515-436ad4e8e681 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:43.444787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de90498-b1c3-4a16-b515-436ad4e8e681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:43.480575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:43.517250 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de90498-b1c3-4a16-b515-436ad4e8e681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:43.518731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de90498-b1c3-4a16-b515-436ad4e8e681 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:43.528682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 964/1895] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:43.789298 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d62ae3-225f-413e-9572-a17e1a6166f2 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:43.810436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:43.861947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d62ae3-225f-413e-9572-a17e1a6166f2 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:43.867586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:43.868525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:43.868812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:43.869089 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] PATH_INFO: `/v3/users/842320d334c143719abd60d48bb53d2f` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:43.870338 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=842320d334c143719abd60d48bb53d2f)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:43.872652 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:43.932520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842320d334c143719abd60d48bb53d2f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:43.940240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842320d334c143719abd60d48bb53d2f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:43.940923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842320d334c143719abd60d48bb53d2f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:43.948136 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842320d334c143719abd60d48bb53d2f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:43.960198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842320d334c143719abd60d48bb53d2f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:43.966856 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 932/1896] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:43 2022] DELETE /identity/v3/users/842320d334c143719abd60d48bb53d2f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:43.971686 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e1d79-0aa8-4f2a-b8f4-354730eb8dbb None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:43.972138 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e1d79-0aa8-4f2a-b8f4-354730eb8dbb None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:43.972285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e1d79-0aa8-4f2a-b8f4-354730eb8dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:43.990463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-896e1d79-0aa8-4f2a-b8f4-354730eb8dbb None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:43.991681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-896e1d79-0aa8-4f2a-b8f4-354730eb8dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:44.058102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 965/1897] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:44.068041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:44.069681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:44.136212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbed42b-a7ad-42a4-9cd9-0dd53b1b13b7 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:44.140392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2b8e3-5891-43a9-b561-874375bcc5c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:44.141416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2b8e3-5891-43a9-b561-874375bcc5c9 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:44.141728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2b8e3-5891-43a9-b561-874375bcc5c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:44.142112 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2b8e3-5891-43a9-b561-874375bcc5c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:44.153733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2b8e3-5891-43a9-b561-874375bcc5c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:44.155192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2b8e3-5891-43a9-b561-874375bcc5c9 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:44.165285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 933/1898] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:44.654654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e1d79-0aa8-4f2a-b8f4-354730eb8dbb None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:44.656188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-896e1d79-0aa8-4f2a-b8f4-354730eb8dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:44.684831 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e1d79-0aa8-4f2a-b8f4-354730eb8dbb None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:44.689585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:44.690537 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:44.690818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:44.691091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] PATH_INFO: `/v3/projects/d849a86eb6f44b2cb5b45bef5170b71a` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:44.692392 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d849a86eb6f44b2cb5b45bef5170b71a)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:44.694335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:44.747929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd849a86eb6f44b2cb5b45bef5170b71a', 'request_id': 'req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:44.755526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd849a86eb6f44b2cb5b45bef5170b71a', 'request_id': 'req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:44.761999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd849a86eb6f44b2cb5b45bef5170b71a', 'request_id': 'req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:44.774521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] The token cache is being invalidate because project d849a86eb6f44b2cb5b45bef5170b71a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:44.774836 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:44.775505 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:44.777140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 966/1899] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:44 2022] DELETE /identity/v3/projects/d849a86eb6f44b2cb5b45bef5170b71a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:44.781158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2b8e3-5891-43a9-b561-874375bcc5c9 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:44.791299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2b8e3-5891-43a9-b561-874375bcc5c9 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:44.830854 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:44.851976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed88a3c-250f-4591-af86-038edd7c3cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:44.852995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed88a3c-250f-4591-af86-038edd7c3cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:44.853269 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed88a3c-250f-4591-af86-038edd7c3cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:44.853542 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed88a3c-250f-4591-af86-038edd7c3cb7 admin admin] PATH_INFO: `/v3/users/5f871bb778e14e0193dc5fa43c090f00` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:44.854842 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed88a3c-250f-4591-af86-038edd7c3cb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f871bb778e14e0193dc5fa43c090f00)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:44.856862 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed88a3c-250f-4591-af86-038edd7c3cb7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:44.911346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ed88a3c-250f-4591-af86-038edd7c3cb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f871bb778e14e0193dc5fa43c090f00'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:44.920634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ed88a3c-250f-4591-af86-038edd7c3cb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f871bb778e14e0193dc5fa43c090f00'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:44.921265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ed88a3c-250f-4591-af86-038edd7c3cb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f871bb778e14e0193dc5fa43c090f00'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:44.929330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ed88a3c-250f-4591-af86-038edd7c3cb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f871bb778e14e0193dc5fa43c090f00'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:44.944389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ed88a3c-250f-4591-af86-038edd7c3cb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f871bb778e14e0193dc5fa43c090f00'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:44.951408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 934/1900] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:44 2022] DELETE /identity/v3/users/5f871bb778e14e0193dc5fa43c090f00 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:45.245199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:45.247538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:45.261732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b57fb-bec2-42a9-9f88-7f13aeebf037 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:45.315287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:45.327067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bd384-adc9-4a83-b61d-f9339eb2493b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:45.328329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bd384-adc9-4a83-b61d-f9339eb2493b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:45.329022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bd384-adc9-4a83-b61d-f9339eb2493b admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:45.329504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bd384-adc9-4a83-b61d-f9339eb2493b admin admin] PATH_INFO: `/v3/projects/d6a4d674c9734956bc5bc3498d15e678` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:45.331068 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1bd384-adc9-4a83-b61d-f9339eb2493b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6a4d674c9734956bc5bc3498d15e678)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:45.333460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1bd384-adc9-4a83-b61d-f9339eb2493b admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:45.391263 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-eb1bd384-adc9-4a83-b61d-f9339eb2493b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6a4d674c9734956bc5bc3498d15e678', 'request_id': 'req-eb1bd384-adc9-4a83-b61d-f9339eb2493b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:45.409301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-eb1bd384-adc9-4a83-b61d-f9339eb2493b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6a4d674c9734956bc5bc3498d15e678', 'request_id': 'req-eb1bd384-adc9-4a83-b61d-f9339eb2493b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:45.418998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-eb1bd384-adc9-4a83-b61d-f9339eb2493b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6a4d674c9734956bc5bc3498d15e678', 'request_id': 'req-eb1bd384-adc9-4a83-b61d-f9339eb2493b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:45.435844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-eb1bd384-adc9-4a83-b61d-f9339eb2493b admin admin] The token cache is being invalidate because project d6a4d674c9734956bc5bc3498d15e678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:45.435844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-eb1bd384-adc9-4a83-b61d-f9339eb2493b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:45.436688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-eb1bd384-adc9-4a83-b61d-f9339eb2493b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:45.451504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 967/1901] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:45 2022] DELETE /identity/v3/projects/d6a4d674c9734956bc5bc3498d15e678 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:45.456588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57a1e8-5e89-4f69-956c-7eda09cad5ec None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:45.457063 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57a1e8-5e89-4f69-956c-7eda09cad5ec None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:45.457471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57a1e8-5e89-4f69-956c-7eda09cad5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:45.481405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8c57a1e8-5e89-4f69-956c-7eda09cad5ec None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:45.482965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8c57a1e8-5e89-4f69-956c-7eda09cad5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:45.649153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 935/1902] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:45.657624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52422285-4584-4a2c-b830-f34ee142c430 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:45.668814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52422285-4584-4a2c-b830-f34ee142c430 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:45.671712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52422285-4584-4a2c-b830-f34ee142c430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:45.727886 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-52422285-4584-4a2c-b830-f34ee142c430 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:45.740577 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-52422285-4584-4a2c-b830-f34ee142c430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:45.769152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 968/1903] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:45.781161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57a1e8-5e89-4f69-956c-7eda09cad5ec None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:45.794966 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57a1e8-5e89-4f69-956c-7eda09cad5ec None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:45.802246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a99a38-03d5-4c2f-aa58-4af98ac999ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:45.802981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a99a38-03d5-4c2f-aa58-4af98ac999ed admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:45.803265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a99a38-03d5-4c2f-aa58-4af98ac999ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:45.803539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a99a38-03d5-4c2f-aa58-4af98ac999ed admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:45.805740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a99a38-03d5-4c2f-aa58-4af98ac999ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:45.806138 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a99a38-03d5-4c2f-aa58-4af98ac999ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:45.808054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a99a38-03d5-4c2f-aa58-4af98ac999ed admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:45.826831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 936/1904] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:45.835776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52422285-4584-4a2c-b830-f34ee142c430 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:45.848919 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52422285-4584-4a2c-b830-f34ee142c430 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:45.856187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321021d-74de-469d-a858-b9a1e0f02161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:45.857268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321021d-74de-469d-a858-b9a1e0f02161 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:45.857907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321021d-74de-469d-a858-b9a1e0f02161 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:45.858213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321021d-74de-469d-a858-b9a1e0f02161 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:45.859871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321021d-74de-469d-a858-b9a1e0f02161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:45.862655 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321021d-74de-469d-a858-b9a1e0f02161 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:45.900470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 969/1905] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:45 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:45.903178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a99a38-03d5-4c2f-aa58-4af98ac999ed admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:45.922448 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a99a38-03d5-4c2f-aa58-4af98ac999ed admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:45.984179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:46.000293 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cceef-e8c0-4ce4-8d77-917a6104db3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.001060 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cceef-e8c0-4ce4-8d77-917a6104db3b admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.001402 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cceef-e8c0-4ce4-8d77-917a6104db3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.001736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cceef-e8c0-4ce4-8d77-917a6104db3b admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.002490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25cceef-e8c0-4ce4-8d77-917a6104db3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.004699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25cceef-e8c0-4ce4-8d77-917a6104db3b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.031875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 937/1906] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:45 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:46.035711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321021d-74de-469d-a858-b9a1e0f02161 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.047419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321021d-74de-469d-a858-b9a1e0f02161 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.052177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbdc93-c3d0-407b-b11e-250ac8942fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.052849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbdc93-c3d0-407b-b11e-250ac8942fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.053138 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbdc93-c3d0-407b-b11e-250ac8942fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.053415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbdc93-c3d0-407b-b11e-250ac8942fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.054137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbdc93-c3d0-407b-b11e-250ac8942fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.055753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfbdc93-c3d0-407b-b11e-250ac8942fe4 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.068595 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 970/1907] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:46.072593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cceef-e8c0-4ce4-8d77-917a6104db3b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.085178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25cceef-e8c0-4ce4-8d77-917a6104db3b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.090075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f96f15-5b21-4583-b95f-245da7c54df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.090753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96f15-5b21-4583-b95f-245da7c54df6 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.091038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96f15-5b21-4583-b95f-245da7c54df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.091313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96f15-5b21-4583-b95f-245da7c54df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.092068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f96f15-5b21-4583-b95f-245da7c54df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.093681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f96f15-5b21-4583-b95f-245da7c54df6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.106107 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 938/1908] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:46.109781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbdc93-c3d0-407b-b11e-250ac8942fe4 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.127102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbdc93-c3d0-407b-b11e-250ac8942fe4 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.134838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73905e26-991e-47f6-879a-b945c728d613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.135752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73905e26-991e-47f6-879a-b945c728d613 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.136058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73905e26-991e-47f6-879a-b945c728d613 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.136336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73905e26-991e-47f6-879a-b945c728d613 admin admin] PATH_INFO: `/v3/projects/3e326983b69846a5a48c16f53980efd6/users/c5089ecbed654648814ce07400439dea/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.151773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73905e26-991e-47f6-879a-b945c728d613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e326983b69846a5a48c16f53980efd6, user_id=c5089ecbed654648814ce07400439dea, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.154578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73905e26-991e-47f6-879a-b945c728d613 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.168682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 971/1909] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:46 2022] PUT /identity/v3/projects/3e326983b69846a5a48c16f53980efd6/users/c5089ecbed654648814ce07400439dea/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:46.180150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af04bf7-2af6-4f10-a6c7-c5e844457b37 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.180616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af04bf7-2af6-4f10-a6c7-c5e844457b37 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.181019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af04bf7-2af6-4f10-a6c7-c5e844457b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.182863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73905e26-991e-47f6-879a-b945c728d613 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.184573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-73905e26-991e-47f6-879a-b945c728d613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:46.208710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7af04bf7-2af6-4f10-a6c7-c5e844457b37 None None] MFA Rules not processed for user `c5089ecbed654648814ce07400439dea`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:46.209822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7af04bf7-2af6-4f10-a6c7-c5e844457b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:46.246610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73905e26-991e-47f6-879a-b945c728d613 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.251548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271cacd-aedf-4547-8e85-9e525b062ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.252482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271cacd-aedf-4547-8e85-9e525b062ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.252802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271cacd-aedf-4547-8e85-9e525b062ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.253040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271cacd-aedf-4547-8e85-9e525b062ee0 admin admin] PATH_INFO: `/v3/users/35655269c1d3479080d10ceec817bbf9` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.254359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271cacd-aedf-4547-8e85-9e525b062ee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35655269c1d3479080d10ceec817bbf9)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.256325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271cacd-aedf-4547-8e85-9e525b062ee0 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.276371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 939/1910] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:46.280951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af04bf7-2af6-4f10-a6c7-c5e844457b37 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.281451 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:46.292994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af04bf7-2af6-4f10-a6c7-c5e844457b37 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.298434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c2830-0c5d-4be4-8844-761e109f1f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.299164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c2830-0c5d-4be4-8844-761e109f1f8e admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.299476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c2830-0c5d-4be4-8844-761e109f1f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.299866 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c2830-0c5d-4be4-8844-761e109f1f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.301055 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c2830-0c5d-4be4-8844-761e109f1f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.303166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c2830-0c5d-4be4-8844-761e109f1f8e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.316890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3271cacd-aedf-4547-8e85-9e525b062ee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35655269c1d3479080d10ceec817bbf9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:46.317591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3271cacd-aedf-4547-8e85-9e525b062ee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35655269c1d3479080d10ceec817bbf9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:46.328662 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3271cacd-aedf-4547-8e85-9e525b062ee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35655269c1d3479080d10ceec817bbf9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:46.335827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 940/1911] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:46 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:46.339905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c2830-0c5d-4be4-8844-761e109f1f8e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.341744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-536c2830-0c5d-4be4-8844-761e109f1f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:46.345758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3271cacd-aedf-4547-8e85-9e525b062ee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35655269c1d3479080d10ceec817bbf9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:46.358249 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3271cacd-aedf-4547-8e85-9e525b062ee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35655269c1d3479080d10ceec817bbf9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:46.365528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 972/1912] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:46 2022] DELETE /identity/v3/users/35655269c1d3479080d10ceec817bbf9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:46.369912 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac5649-44c4-45d4-ae16-db2178a4abeb None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.370242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac5649-44c4-45d4-ae16-db2178a4abeb None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.370956 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac5649-44c4-45d4-ae16-db2178a4abeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.388938 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-fcac5649-44c4-45d4-ae16-db2178a4abeb None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:46.390143 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-fcac5649-44c4-45d4-ae16-db2178a4abeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:46.408854 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c2830-0c5d-4be4-8844-761e109f1f8e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.448925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:46.449931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 973/1913] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:46.459915 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac5649-44c4-45d4-ae16-db2178a4abeb None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.461319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-fcac5649-44c4-45d4-ae16-db2178a4abeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:46.469823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7b6bf-387c-48fb-8766-c463d325a68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.470509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7b6bf-387c-48fb-8766-c463d325a68e admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.470788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7b6bf-387c-48fb-8766-c463d325a68e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.471059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7b6bf-387c-48fb-8766-c463d325a68e admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.471748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7b6bf-387c-48fb-8766-c463d325a68e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.473758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7b6bf-387c-48fb-8766-c463d325a68e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.496079 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 941/1914] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:46 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:46.498534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7b6bf-387c-48fb-8766-c463d325a68e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.509287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7b6bf-387c-48fb-8766-c463d325a68e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.513933 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86991930-674d-4eb6-bc95-2aea50f69871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.514598 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86991930-674d-4eb6-bc95-2aea50f69871 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.514885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86991930-674d-4eb6-bc95-2aea50f69871 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.515169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86991930-674d-4eb6-bc95-2aea50f69871 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.515909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86991930-674d-4eb6-bc95-2aea50f69871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.517546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86991930-674d-4eb6-bc95-2aea50f69871 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.519996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac5649-44c4-45d4-ae16-db2178a4abeb None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.524505 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d5978-7990-4dda-9d73-ccadbb1cf915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.525345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d5978-7990-4dda-9d73-ccadbb1cf915 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.525567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d5978-7990-4dda-9d73-ccadbb1cf915 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.525940 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d5978-7990-4dda-9d73-ccadbb1cf915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.529998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 942/1915] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:49:46.534363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86991930-674d-4eb6-bc95-2aea50f69871 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.538792 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d5978-7990-4dda-9d73-ccadbb1cf915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.540333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d5978-7990-4dda-9d73-ccadbb1cf915 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.544282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86991930-674d-4eb6-bc95-2aea50f69871 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.549553 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac208019-642d-46ff-871d-f2352721f8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.550439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac208019-642d-46ff-871d-f2352721f8a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.550723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac208019-642d-46ff-871d-f2352721f8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.550829 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 974/1916] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:49:46.551113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac208019-642d-46ff-871d-f2352721f8a3 admin admin] PATH_INFO: `/v3/projects/431458e6d9e645dfa330ab518f2aec20/users/7ff5234139cb4b808e43b0e6e7d51ee3/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.567553 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac208019-642d-46ff-871d-f2352721f8a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=431458e6d9e645dfa330ab518f2aec20, user_id=7ff5234139cb4b808e43b0e6e7d51ee3, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.570648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac208019-642d-46ff-871d-f2352721f8a3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.586241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 943/1917] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:46 2022] PUT /identity/v3/projects/431458e6d9e645dfa330ab518f2aec20/users/7ff5234139cb4b808e43b0e6e7d51ee3/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:46.590213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d5978-7990-4dda-9d73-ccadbb1cf915 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.600427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d5978-7990-4dda-9d73-ccadbb1cf915 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.650801 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:46.676244 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143340e2-f707-44a1-ba77-4b2da1ef165d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.677104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143340e2-f707-44a1-ba77-4b2da1ef165d admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.677391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143340e2-f707-44a1-ba77-4b2da1ef165d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.677667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143340e2-f707-44a1-ba77-4b2da1ef165d admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.678467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143340e2-f707-44a1-ba77-4b2da1ef165d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.680179 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143340e2-f707-44a1-ba77-4b2da1ef165d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.693483 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 975/1918] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:46.698150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac208019-642d-46ff-871d-f2352721f8a3 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.713548 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac208019-642d-46ff-871d-f2352721f8a3 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.718976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7b52f-2923-42f9-b6d0-363f952c8176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.719323 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7b52f-2923-42f9-b6d0-363f952c8176 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.719554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7b52f-2923-42f9-b6d0-363f952c8176 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.719828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7b52f-2923-42f9-b6d0-363f952c8176 admin admin] PATH_INFO: `/v3/projects/431458e6d9e645dfa330ab518f2aec20/users/7ff5234139cb4b808e43b0e6e7d51ee3/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.721958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7b52f-2923-42f9-b6d0-363f952c8176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=431458e6d9e645dfa330ab518f2aec20, user_id=7ff5234139cb4b808e43b0e6e7d51ee3, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.724839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7b52f-2923-42f9-b6d0-363f952c8176 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.750572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 944/1919] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:46 2022] PUT /identity/v3/projects/431458e6d9e645dfa330ab518f2aec20/users/7ff5234139cb4b808e43b0e6e7d51ee3/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:46.761517 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9930c0-433d-4c98-91e1-be6f8b9b40df None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.761807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9930c0-433d-4c98-91e1-be6f8b9b40df None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.762085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9930c0-433d-4c98-91e1-be6f8b9b40df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.790636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7b52f-2923-42f9-b6d0-363f952c8176 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.793147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a4d7b52f-2923-42f9-b6d0-363f952c8176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:46.798329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-ea9930c0-433d-4c98-91e1-be6f8b9b40df None None] MFA Rules not processed for user `7ff5234139cb4b808e43b0e6e7d51ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:46.799556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ea9930c0-433d-4c98-91e1-be6f8b9b40df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:46.871982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7b52f-2923-42f9-b6d0-363f952c8176 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.879982 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 976/1920] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:46.880175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.880175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.880175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.880175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] PATH_INFO: `/v3/users/1fbaf5030dfa409bb52b7d0b737bdb99` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.880452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fbaf5030dfa409bb52b7d0b737bdb99)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:46.888046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9930c0-433d-4c98-91e1-be6f8b9b40df None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:46.888612 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:46.889532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ea9930c0-433d-4c98-91e1-be6f8b9b40df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:46.951513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fbaf5030dfa409bb52b7d0b737bdb99'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:46.967961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fbaf5030dfa409bb52b7d0b737bdb99'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:46.969012 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fbaf5030dfa409bb52b7d0b737bdb99'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:46.976320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9930c0-433d-4c98-91e1-be6f8b9b40df None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:46.979432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fbaf5030dfa409bb52b7d0b737bdb99'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:46.980288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e0b8a-726a-4651-9d89-97233a4d2a84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:46.981657 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e0b8a-726a-4651-9d89-97233a4d2a84 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:46.981947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e0b8a-726a-4651-9d89-97233a4d2a84 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:46.982226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e0b8a-726a-4651-9d89-97233a4d2a84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:46.984202 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e87e0b8a-726a-4651-9d89-97233a4d2a84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:47.023529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fbaf5030dfa409bb52b7d0b737bdb99'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.027353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:47.032480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e0b8a-726a-4651-9d89-97233a4d2a84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:47.034243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e0b8a-726a-4651-9d89-97233a4d2a84 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:47.037523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 945/1921] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:46 2022] DELETE /identity/v3/users/1fbaf5030dfa409bb52b7d0b737bdb99 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:47.037523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:47.040288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:47.057091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 977/1922] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:47.059213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c86abe-bd3f-4ea0-913d-0174424c4f02 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:47.059511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c86abe-bd3f-4ea0-913d-0174424c4f02 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:47.059788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c86abe-bd3f-4ea0-913d-0174424c4f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:47.085260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-44c86abe-bd3f-4ea0-913d-0174424c4f02 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:47.086881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-44c86abe-bd3f-4ea0-913d-0174424c4f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:47.169887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69152b0f-db35-414f-89d7-a0767cf68eb6 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:47.175531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:47.176768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:47.177574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:47.178024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] PATH_INFO: `/v3/projects/28fae6efdcd447baa6a9df2c36bb976d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:47.180286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28fae6efdcd447baa6a9df2c36bb976d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:47.182906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:47.204381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 978/1923] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:47.204381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c86abe-bd3f-4ea0-913d-0174424c4f02 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:47.218029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-44c86abe-bd3f-4ea0-913d-0174424c4f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:47.233993 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c86abe-bd3f-4ea0-913d-0174424c4f02 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:47.247554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:47.249533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28fae6efdcd447baa6a9df2c36bb976d', 'request_id': 'req-ebe719f8-f33c-4c27-b857-fa477fbf7c32'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.258356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28fae6efdcd447baa6a9df2c36bb976d', 'request_id': 'req-ebe719f8-f33c-4c27-b857-fa477fbf7c32'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.268240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28fae6efdcd447baa6a9df2c36bb976d', 'request_id': 'req-ebe719f8-f33c-4c27-b857-fa477fbf7c32'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.280939 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] The token cache is being invalidate because project 28fae6efdcd447baa6a9df2c36bb976d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:47.281224 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.281882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.284221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 946/1924] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:47 2022] DELETE /identity/v3/projects/28fae6efdcd447baa6a9df2c36bb976d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:47.284809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:47.288302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:47.307137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af084d-e6ff-424a-998f-0c3324d10313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:47.308486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af084d-e6ff-424a-998f-0c3324d10313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:47.308781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af084d-e6ff-424a-998f-0c3324d10313 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:47.309056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af084d-e6ff-424a-998f-0c3324d10313 admin admin] PATH_INFO: `/v3/users/c5089ecbed654648814ce07400439dea` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:47.310720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af084d-e6ff-424a-998f-0c3324d10313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5089ecbed654648814ce07400439dea)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:47.313035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af084d-e6ff-424a-998f-0c3324d10313 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:47.355442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe719f8-f33c-4c27-b857-fa477fbf7c32 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:47.359199 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9361e3-1ed9-4566-805f-ecb2fa61066b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:47.359879 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9361e3-1ed9-4566-805f-ecb2fa61066b service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:47.360178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9361e3-1ed9-4566-805f-ecb2fa61066b service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:47.360454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9361e3-1ed9-4566-805f-ecb2fa61066b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:47.362325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ff9361e3-1ed9-4566-805f-ecb2fa61066b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:47.368704 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-39af084d-e6ff-424a-998f-0c3324d10313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5089ecbed654648814ce07400439dea'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.369239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-39af084d-e6ff-424a-998f-0c3324d10313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5089ecbed654648814ce07400439dea'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.375646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9361e3-1ed9-4566-805f-ecb2fa61066b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:47.377772 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9361e3-1ed9-4566-805f-ecb2fa61066b service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:47.380501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-39af084d-e6ff-424a-998f-0c3324d10313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5089ecbed654648814ce07400439dea'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.389163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-39af084d-e6ff-424a-998f-0c3324d10313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5089ecbed654648814ce07400439dea'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.391889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 947/1925] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:47.392776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9361e3-1ed9-4566-805f-ecb2fa61066b service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:47.394152 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ff9361e3-1ed9-4566-805f-ecb2fa61066b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:47.408261 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9361e3-1ed9-4566-805f-ecb2fa61066b service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:47.410926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-39af084d-e6ff-424a-998f-0c3324d10313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5089ecbed654648814ce07400439dea'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.413661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:47.415071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:47.415574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:47.415994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] PATH_INFO: `/v3/users/1c871a7e51634971b5e963d3550df171` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:47.417268 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c871a7e51634971b5e963d3550df171)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:47.421556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 979/1926] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:47 2022] DELETE /identity/v3/users/c5089ecbed654648814ce07400439dea => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:47.421788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:47.423705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5329e1b-1c25-46a8-aaaf-5c3298a9da26 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:47.424961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5329e1b-1c25-46a8-aaaf-5c3298a9da26 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:47.425336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5329e1b-1c25-46a8-aaaf-5c3298a9da26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:47.467020 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d5329e1b-1c25-46a8-aaaf-5c3298a9da26 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:47.468610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d5329e1b-1c25-46a8-aaaf-5c3298a9da26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:47.472985 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:47.501430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c871a7e51634971b5e963d3550df171'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.511950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c871a7e51634971b5e963d3550df171'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.514889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c871a7e51634971b5e963d3550df171'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.537955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c871a7e51634971b5e963d3550df171'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.560445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c871a7e51634971b5e963d3550df171'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:47.570041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 948/1927] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:47 2022] DELETE /identity/v3/users/1c871a7e51634971b5e963d3550df171 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:47.612658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 980/1928] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:47.629647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:47.631505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:47.760336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ed2de-3880-4d63-b5bc-83bd924dbbeb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:47.769236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1058b83-6a21-47fb-9f67-0039d5fe52d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:47.771258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1058b83-6a21-47fb-9f67-0039d5fe52d1 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:47.771258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1058b83-6a21-47fb-9f67-0039d5fe52d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:47.771681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1058b83-6a21-47fb-9f67-0039d5fe52d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:47.777089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5329e1b-1c25-46a8-aaaf-5c3298a9da26 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:47.779629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d5329e1b-1c25-46a8-aaaf-5c3298a9da26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:47.813226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5329e1b-1c25-46a8-aaaf-5c3298a9da26 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:47.881408 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:47.897160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1058b83-6a21-47fb-9f67-0039d5fe52d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:47.898914 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1058b83-6a21-47fb-9f67-0039d5fe52d1 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:47.910194 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 949/1929] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:47.941168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:47.942682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:47.942971 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:47.943247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] PATH_INFO: `/v3/projects/fdc613edd261411fa7c9352fd2d0285e` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:47.944490 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc613edd261411fa7c9352fd2d0285e)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:47.946645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:48.032129 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc613edd261411fa7c9352fd2d0285e', 'request_id': 'req-f22dd46e-c6fe-4229-937b-e388a3d62b49'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.042378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc613edd261411fa7c9352fd2d0285e', 'request_id': 'req-f22dd46e-c6fe-4229-937b-e388a3d62b49'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.052990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc613edd261411fa7c9352fd2d0285e', 'request_id': 'req-f22dd46e-c6fe-4229-937b-e388a3d62b49'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.068882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] The token cache is being invalidate because project fdc613edd261411fa7c9352fd2d0285e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:48.069215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.070170 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.072061 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 981/1930] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:47 2022] DELETE /identity/v3/projects/fdc613edd261411fa7c9352fd2d0285e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:48.076414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1058b83-6a21-47fb-9f67-0039d5fe52d1 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:48.091987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1058b83-6a21-47fb-9f67-0039d5fe52d1 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:48.166581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:48.174181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:48.202653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:48.230124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7610a1-abe4-4df6-aa1b-c9e6739411c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:48.232253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7610a1-abe4-4df6-aa1b-c9e6739411c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:48.233600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7610a1-abe4-4df6-aa1b-c9e6739411c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:48.234257 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7610a1-abe4-4df6-aa1b-c9e6739411c7 admin admin] PATH_INFO: `/v3/users/cf436cab698945c58a52595c5201b114` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:48.237857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7610a1-abe4-4df6-aa1b-c9e6739411c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf436cab698945c58a52595c5201b114)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:48.242900 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7610a1-abe4-4df6-aa1b-c9e6739411c7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:48.321716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dd46e-c6fe-4229-937b-e388a3d62b49 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:48.341563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:48.348383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:48.348681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:48.348965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] PATH_INFO: `/v3/projects/497bd535a4b04d668f1cec811c136a3d` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:48.354004 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=497bd535a4b04d668f1cec811c136a3d)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:48.356899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:48.358279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-bf7610a1-abe4-4df6-aa1b-c9e6739411c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf436cab698945c58a52595c5201b114'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.372878 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-bf7610a1-abe4-4df6-aa1b-c9e6739411c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf436cab698945c58a52595c5201b114'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.373634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-bf7610a1-abe4-4df6-aa1b-c9e6739411c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf436cab698945c58a52595c5201b114'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.386494 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-bf7610a1-abe4-4df6-aa1b-c9e6739411c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf436cab698945c58a52595c5201b114'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.409503 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-bf7610a1-abe4-4df6-aa1b-c9e6739411c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf436cab698945c58a52595c5201b114'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.420091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 950/1931] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:48 2022] DELETE /identity/v3/users/cf436cab698945c58a52595c5201b114 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:48.433724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '497bd535a4b04d668f1cec811c136a3d', 'request_id': 'req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.447231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '497bd535a4b04d668f1cec811c136a3d', 'request_id': 'req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.450869 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:48.461985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '497bd535a4b04d668f1cec811c136a3d', 'request_id': 'req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.474853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] The token cache is being invalidate because project 497bd535a4b04d668f1cec811c136a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:48.475145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.475804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.477714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 982/1932] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:48 2022] DELETE /identity/v3/projects/497bd535a4b04d668f1cec811c136a3d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:48.482884 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7209d-2f0d-448f-9b13-b8a6f072a88c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:48.483322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7209d-2f0d-448f-9b13-b8a6f072a88c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:48.484058 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7209d-2f0d-448f-9b13-b8a6f072a88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:48.488723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:48.494065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:48.525421 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-faa7209d-2f0d-448f-9b13-b8a6f072a88c None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:48.529380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-faa7209d-2f0d-448f-9b13-b8a6f072a88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:48.638077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3fdd3-f12c-46d4-9fa8-a7d2870f6e50 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:48.655473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:48.656604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:48.656897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:48.657180 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] PATH_INFO: `/v3/projects/3e326983b69846a5a48c16f53980efd6` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:48.658881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e326983b69846a5a48c16f53980efd6)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:48.662173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:48.690674 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 951/1933] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:48.699010 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc52be1-3f1e-414a-8594-f367e130fa39 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:48.700418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc52be1-3f1e-414a-8594-f367e130fa39 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:48.701144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc52be1-3f1e-414a-8594-f367e130fa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:48.730560 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e326983b69846a5a48c16f53980efd6', 'request_id': 'req-642e2c9a-987e-4b02-9a53-b0186f6cbd41'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.731087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ffc52be1-3f1e-414a-8594-f367e130fa39 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:48.733198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ffc52be1-3f1e-414a-8594-f367e130fa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:48.743977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e326983b69846a5a48c16f53980efd6', 'request_id': 'req-642e2c9a-987e-4b02-9a53-b0186f6cbd41'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.754573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e326983b69846a5a48c16f53980efd6', 'request_id': 'req-642e2c9a-987e-4b02-9a53-b0186f6cbd41'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.771323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] The token cache is being invalidate because project 3e326983b69846a5a48c16f53980efd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:48.771618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.772328 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:48.777621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 983/1934] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:48 2022] DELETE /identity/v3/projects/3e326983b69846a5a48c16f53980efd6 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:48.779121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:48.781614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:48.856834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 952/1935] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:49:48.861331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc52be1-3f1e-414a-8594-f367e130fa39 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:48.874636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc52be1-3f1e-414a-8594-f367e130fa39 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:48.881130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa676c-b3d3-4d1e-9010-c25fcd9f75e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:48.882445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa676c-b3d3-4d1e-9010-c25fcd9f75e8 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:48.883486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa676c-b3d3-4d1e-9010-c25fcd9f75e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:48.883837 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa676c-b3d3-4d1e-9010-c25fcd9f75e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:48.885751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa676c-b3d3-4d1e-9010-c25fcd9f75e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:48.886164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa676c-b3d3-4d1e-9010-c25fcd9f75e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:48.888893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa676c-b3d3-4d1e-9010-c25fcd9f75e8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:48.891443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e2c9a-987e-4b02-9a53-b0186f6cbd41 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:48.901249 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:48.904554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:48.904853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:48.905600 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] PATH_INFO: `/v3/users/7ff5234139cb4b808e43b0e6e7d51ee3` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:48.908195 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 953/1936] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:48.908875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ff5234139cb4b808e43b0e6e7d51ee3)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:48.912186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa676c-b3d3-4d1e-9010-c25fcd9f75e8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:48.913277 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:48.923716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:48.938827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa676c-b3d3-4d1e-9010-c25fcd9f75e8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:48.952633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce2b24-366f-4a64-be8a-0b8270dcd428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:48.953326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce2b24-366f-4a64-be8a-0b8270dcd428 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:48.953756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce2b24-366f-4a64-be8a-0b8270dcd428 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:48.954026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce2b24-366f-4a64-be8a-0b8270dcd428 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:48.954717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce2b24-366f-4a64-be8a-0b8270dcd428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:48.957063 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce2b24-366f-4a64-be8a-0b8270dcd428 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:48.982771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:49.007287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 954/1937] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:48 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:49.011206 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce2b24-366f-4a64-be8a-0b8270dcd428 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:49.024306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ff5234139cb4b808e43b0e6e7d51ee3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:49.024983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ff5234139cb4b808e43b0e6e7d51ee3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:49.037845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ff5234139cb4b808e43b0e6e7d51ee3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:49.055799 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ff5234139cb4b808e43b0e6e7d51ee3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:49.086623 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ff5234139cb4b808e43b0e6e7d51ee3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:49.093796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 984/1938] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:48 2022] DELETE /identity/v3/users/7ff5234139cb4b808e43b0e6e7d51ee3 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:49.103914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:49.105724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:49.107305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce2b24-366f-4a64-be8a-0b8270dcd428 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:49.194355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d61aa-ec73-41bc-aacb-d3e2b406f9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:49.205316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d61aa-ec73-41bc-aacb-d3e2b406f9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:49.205316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d61aa-ec73-41bc-aacb-d3e2b406f9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:49.205316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d61aa-ec73-41bc-aacb-d3e2b406f9b2 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:49.205316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d61aa-ec73-41bc-aacb-d3e2b406f9b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:49.205316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d61aa-ec73-41bc-aacb-d3e2b406f9b2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:49.205811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b51743-a3cb-4928-a337-253e519ed730 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:49.208631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea436ae-5454-48d4-8598-c52164dca3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:49.209530 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea436ae-5454-48d4-8598-c52164dca3d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:49.209813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea436ae-5454-48d4-8598-c52164dca3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:49.210084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea436ae-5454-48d4-8598-c52164dca3d7 admin admin] PATH_INFO: `/v3/projects/fd2acb1ca0654938996b0b2722bf6153` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:49.211448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea436ae-5454-48d4-8598-c52164dca3d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd2acb1ca0654938996b0b2722bf6153)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:49.215471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea436ae-5454-48d4-8598-c52164dca3d7 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:49.245948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 955/1939] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:49 2022] POST /identity/v3/users => generated 291 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:49:49.247558 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d61aa-ec73-41bc-aacb-d3e2b406f9b2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:49.249315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-eb5d61aa-ec73-41bc-aacb-d3e2b406f9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:49.263597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d61aa-ec73-41bc-aacb-d3e2b406f9b2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:49.271073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe67ee-ee52-4c89-8927-9e24dcbdb0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:49.272868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe67ee-ee52-4c89-8927-9e24dcbdb0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:49.273368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe67ee-ee52-4c89-8927-9e24dcbdb0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:49.273847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe67ee-ee52-4c89-8927-9e24dcbdb0f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:49.274904 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe67ee-ee52-4c89-8927-9e24dcbdb0f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:49.282968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe67ee-ee52-4c89-8927-9e24dcbdb0f0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:49.293718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9ea436ae-5454-48d4-8598-c52164dca3d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd2acb1ca0654938996b0b2722bf6153', 'request_id': 'req-9ea436ae-5454-48d4-8598-c52164dca3d7'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:49.302734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 956/1940] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:49:49.304986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe67ee-ee52-4c89-8927-9e24dcbdb0f0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:49.305254 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9ea436ae-5454-48d4-8598-c52164dca3d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd2acb1ca0654938996b0b2722bf6153', 'request_id': 'req-9ea436ae-5454-48d4-8598-c52164dca3d7'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:49.314317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9ea436ae-5454-48d4-8598-c52164dca3d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd2acb1ca0654938996b0b2722bf6153', 'request_id': 'req-9ea436ae-5454-48d4-8598-c52164dca3d7'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:49.319446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe67ee-ee52-4c89-8927-9e24dcbdb0f0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:49.328746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9ea436ae-5454-48d4-8598-c52164dca3d7 admin admin] The token cache is being invalidate because project fd2acb1ca0654938996b0b2722bf6153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:49.329337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9ea436ae-5454-48d4-8598-c52164dca3d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:49.329771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9ea436ae-5454-48d4-8598-c52164dca3d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:49.336116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 985/1941] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:49 2022] DELETE /identity/v3/projects/fd2acb1ca0654938996b0b2722bf6153 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:49.338224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bf2b0-3369-4ddf-a201-06e99864dca2 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:49.338400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bf2b0-3369-4ddf-a201-06e99864dca2 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:49.338703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bf2b0-3369-4ddf-a201-06e99864dca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:49.360182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-108bf2b0-3369-4ddf-a201-06e99864dca2 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:49.362040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-108bf2b0-3369-4ddf-a201-06e99864dca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:49.387848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:49.413354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59529e1-b89d-4405-88ca-3f867ea79921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:49.414813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59529e1-b89d-4405-88ca-3f867ea79921 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:49.415099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59529e1-b89d-4405-88ca-3f867ea79921 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:49.415758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59529e1-b89d-4405-88ca-3f867ea79921 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:49.417811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59529e1-b89d-4405-88ca-3f867ea79921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:49.422230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59529e1-b89d-4405-88ca-3f867ea79921 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:49.449677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 957/1942] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:49.450740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59529e1-b89d-4405-88ca-3f867ea79921 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:49.453719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e59529e1-b89d-4405-88ca-3f867ea79921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:49.464873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:49.471606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 986/1943] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:49.479793 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4db39-8d21-4d4c-aa4a-6a0dea381a6b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:49.483588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4db39-8d21-4d4c-aa4a-6a0dea381a6b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:49.484832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59529e1-b89d-4405-88ca-3f867ea79921 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:49.485682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4db39-8d21-4d4c-aa4a-6a0dea381a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:49.491154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b7807-00ae-4414-bc15-3c459c34426f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:49.492093 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b7807-00ae-4414-bc15-3c459c34426f admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:49.492383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b7807-00ae-4414-bc15-3c459c34426f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:49.493688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b7807-00ae-4414-bc15-3c459c34426f admin admin] PATH_INFO: `/v3/projects/946175c6149b46b1838d8ebb93efba66/users/809518ef65fb410ea8b522b13798a87c/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:49.510694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b7807-00ae-4414-bc15-3c459c34426f admin admin] RBAC: Authorizing `identity:create_grant(project_id=946175c6149b46b1838d8ebb93efba66, user_id=809518ef65fb410ea8b522b13798a87c, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:49.515275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b7807-00ae-4414-bc15-3c459c34426f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:49.530491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-5fa4db39-8d21-4d4c-aa4a-6a0dea381a6b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:49.532680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5fa4db39-8d21-4d4c-aa4a-6a0dea381a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:49.536107 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 958/1944] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:49 2022] PUT /identity/v3/projects/946175c6149b46b1838d8ebb93efba66/users/809518ef65fb410ea8b522b13798a87c/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:49.547839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d4559-b0cb-4cf1-a32f-8791cb7b0f16 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:49.548334 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d4559-b0cb-4cf1-a32f-8791cb7b0f16 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:49.548755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d4559-b0cb-4cf1-a32f-8791cb7b0f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:49.611786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8e1d4559-b0cb-4cf1-a32f-8791cb7b0f16 None None] MFA Rules not processed for user `809518ef65fb410ea8b522b13798a87c`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:49.613260 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8e1d4559-b0cb-4cf1-a32f-8791cb7b0f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:49.744400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 959/1945] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:49.746077 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d4559-b0cb-4cf1-a32f-8791cb7b0f16 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:49.747746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 987/1946] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:49.752376 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4db39-8d21-4d4c-aa4a-6a0dea381a6b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:49.761889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d4559-b0cb-4cf1-a32f-8791cb7b0f16 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:49.769339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ded8d2-ac0d-4257-a7b6-cdfeedfdca9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:49.770840 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ded8d2-ac0d-4257-a7b6-cdfeedfdca9f admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:49.771365 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ded8d2-ac0d-4257-a7b6-cdfeedfdca9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:49.771848 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ded8d2-ac0d-4257-a7b6-cdfeedfdca9f admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:49.772919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ded8d2-ac0d-4257-a7b6-cdfeedfdca9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:49.773404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ded8d2-ac0d-4257-a7b6-cdfeedfdca9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:49.774393 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4db39-8d21-4d4c-aa4a-6a0dea381a6b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:49.775695 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ded8d2-ac0d-4257-a7b6-cdfeedfdca9f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:49.790987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c071a-7771-4738-a18f-6a9144fae5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:49.792262 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c071a-7771-4738-a18f-6a9144fae5b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:49.793716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 960/1947] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:49.794563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c071a-7771-4738-a18f-6a9144fae5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:49.794890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c071a-7771-4738-a18f-6a9144fae5b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:49.796925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c071a-7771-4738-a18f-6a9144fae5b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:49.798978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ded8d2-ac0d-4257-a7b6-cdfeedfdca9f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:49.803036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c071a-7771-4738-a18f-6a9144fae5b2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:49.811261 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ded8d2-ac0d-4257-a7b6-cdfeedfdca9f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:49.818182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32db2fa-5615-4021-8e3a-1a6262da9304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:49.818796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32db2fa-5615-4021-8e3a-1a6262da9304 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:49.819079 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32db2fa-5615-4021-8e3a-1a6262da9304 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:49.819348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32db2fa-5615-4021-8e3a-1a6262da9304 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:49.820194 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32db2fa-5615-4021-8e3a-1a6262da9304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:49.822926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32db2fa-5615-4021-8e3a-1a6262da9304 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:49.867577 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 988/1948] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:49 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:49.868798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c071a-7771-4738-a18f-6a9144fae5b2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:49.873348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 961/1949] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:49 2022] POST /identity/v3/projects => generated 409 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:49.874305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32db2fa-5615-4021-8e3a-1a6262da9304 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:49.874820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f87c071a-7771-4738-a18f-6a9144fae5b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:49.950094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32db2fa-5615-4021-8e3a-1a6262da9304 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:49.983829 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c071a-7771-4738-a18f-6a9144fae5b2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:49.988429 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:49.989342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:49.989624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:49.989904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] PATH_INFO: `/v3/users/75647c03a37944979657ac3ecd6cc5d2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:49.990115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:49.991096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=75647c03a37944979657ac3ecd6cc5d2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:49.993090 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.009884 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5c4ef-f8aa-4961-adaa-717b315335ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.010572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5c4ef-f8aa-4961-adaa-717b315335ea admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.010859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5c4ef-f8aa-4961-adaa-717b315335ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.011137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5c4ef-f8aa-4961-adaa-717b315335ea admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.011829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5c4ef-f8aa-4961-adaa-717b315335ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.013828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5c4ef-f8aa-4961-adaa-717b315335ea admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.034545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 962/1950] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:49 2022] POST /identity/v3/users => generated 305 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:50.035398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5c4ef-f8aa-4961-adaa-717b315335ea admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.041941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75647c03a37944979657ac3ecd6cc5d2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:50.042631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75647c03a37944979657ac3ecd6cc5d2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:50.044631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:50.045019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5c4ef-f8aa-4961-adaa-717b315335ea admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.050254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e249637-cb72-4b89-b271-74e87db291b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.050945 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e249637-cb72-4b89-b271-74e87db291b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.051231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e249637-cb72-4b89-b271-74e87db291b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.051507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e249637-cb72-4b89-b271-74e87db291b7 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.052156 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75647c03a37944979657ac3ecd6cc5d2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:50.052266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e249637-cb72-4b89-b271-74e87db291b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.054150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e249637-cb72-4b89-b271-74e87db291b7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.060269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75647c03a37944979657ac3ecd6cc5d2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:50.073345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75647c03a37944979657ac3ecd6cc5d2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:50.074877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 963/1951] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:50 2022] POST /identity/v3/users => generated 297 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:50.075917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e249637-cb72-4b89-b271-74e87db291b7 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.078435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6e249637-cb72-4b89-b271-74e87db291b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:50.081125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 989/1952] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:49 2022] DELETE /identity/v3/users/75647c03a37944979657ac3ecd6cc5d2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:50.088735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.090987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:50.093124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e249637-cb72-4b89-b271-74e87db291b7 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.158501 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38865664-8059-4a55-bc3c-7cd18dc150cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.159704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38865664-8059-4a55-bc3c-7cd18dc150cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.160139 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38865664-8059-4a55-bc3c-7cd18dc150cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.160524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38865664-8059-4a55-bc3c-7cd18dc150cc admin admin] PATH_INFO: `/v3/projects/431458e6d9e645dfa330ab518f2aec20` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.161945 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38865664-8059-4a55-bc3c-7cd18dc150cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=431458e6d9e645dfa330ab518f2aec20)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.164119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38865664-8059-4a55-bc3c-7cd18dc150cc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.164425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33733d-0fbf-423f-a400-ea5fe44fdfbd admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.169157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf334c-4ac4-4840-ae7c-aa9f16ed1c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.169841 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf334c-4ac4-4840-ae7c-aa9f16ed1c57 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.170121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf334c-4ac4-4840-ae7c-aa9f16ed1c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.170392 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf334c-4ac4-4840-ae7c-aa9f16ed1c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.171279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf334c-4ac4-4840-ae7c-aa9f16ed1c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.172930 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf334c-4ac4-4840-ae7c-aa9f16ed1c57 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.186411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 990/1953] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:50.190748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf334c-4ac4-4840-ae7c-aa9f16ed1c57 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.196764 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-9cbf334c-4ac4-4840-ae7c-aa9f16ed1c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:50.211977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf334c-4ac4-4840-ae7c-aa9f16ed1c57 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.226105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-38865664-8059-4a55-bc3c-7cd18dc150cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '431458e6d9e645dfa330ab518f2aec20', 'request_id': 'req-38865664-8059-4a55-bc3c-7cd18dc150cc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:50.234700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-38865664-8059-4a55-bc3c-7cd18dc150cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '431458e6d9e645dfa330ab518f2aec20', 'request_id': 'req-38865664-8059-4a55-bc3c-7cd18dc150cc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:50.241365 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-38865664-8059-4a55-bc3c-7cd18dc150cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '431458e6d9e645dfa330ab518f2aec20', 'request_id': 'req-38865664-8059-4a55-bc3c-7cd18dc150cc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:50.254266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-38865664-8059-4a55-bc3c-7cd18dc150cc admin admin] The token cache is being invalidate because project 431458e6d9e645dfa330ab518f2aec20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:50.254552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-38865664-8059-4a55-bc3c-7cd18dc150cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:50.255220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-38865664-8059-4a55-bc3c-7cd18dc150cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:50.256842 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 964/1954] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:50 2022] DELETE /identity/v3/projects/431458e6d9e645dfa330ab518f2aec20 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:50.258105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc38a1-a8bf-4e34-a71c-76a6aa67da45 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.258389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc38a1-a8bf-4e34-a71c-76a6aa67da45 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.258665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc38a1-a8bf-4e34-a71c-76a6aa67da45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.282231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507de64-fe5c-440f-b879-cd3e75ead96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.282975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507de64-fe5c-440f-b879-cd3e75ead96d admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.283203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507de64-fe5c-440f-b879-cd3e75ead96d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.283480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507de64-fe5c-440f-b879-cd3e75ead96d admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.283955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-51dc38a1-a8bf-4e34-a71c-76a6aa67da45 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:50.284245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507de64-fe5c-440f-b879-cd3e75ead96d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.285248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-51dc38a1-a8bf-4e34-a71c-76a6aa67da45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:50.285936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507de64-fe5c-440f-b879-cd3e75ead96d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.293488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 965/1955] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:50.294234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc38a1-a8bf-4e34-a71c-76a6aa67da45 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.295484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-51dc38a1-a8bf-4e34-a71c-76a6aa67da45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:50.297084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 991/1956] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:50.298589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b3c60-7431-4262-92a7-4ccff3460cc9 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.298893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b3c60-7431-4262-92a7-4ccff3460cc9 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.299171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b3c60-7431-4262-92a7-4ccff3460cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.309688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc38a1-a8bf-4e34-a71c-76a6aa67da45 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.314763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691350bc-691d-4798-8cdf-726664d2aa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.315444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691350bc-691d-4798-8cdf-726664d2aa94 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.315740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691350bc-691d-4798-8cdf-726664d2aa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.316030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691350bc-691d-4798-8cdf-726664d2aa94 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.316823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691350bc-691d-4798-8cdf-726664d2aa94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.318422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691350bc-691d-4798-8cdf-726664d2aa94 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.318997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-575b3c60-7431-4262-92a7-4ccff3460cc9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:50.321101 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-575b3c60-7431-4262-92a7-4ccff3460cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:50.331521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 992/1957] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:50.332419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b3c60-7431-4262-92a7-4ccff3460cc9 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.333785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-575b3c60-7431-4262-92a7-4ccff3460cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:50.339591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 966/1958] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:50.339591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691350bc-691d-4798-8cdf-726664d2aa94 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.340356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-691350bc-691d-4798-8cdf-726664d2aa94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:50.348871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b3c60-7431-4262-92a7-4ccff3460cc9 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.353963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4fd23-5e3d-430d-b9d2-81162e57001c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.354863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4fd23-5e3d-430d-b9d2-81162e57001c admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.355148 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4fd23-5e3d-430d-b9d2-81162e57001c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.355426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4fd23-5e3d-430d-b9d2-81162e57001c admin admin] PATH_INFO: `/v3/projects/ded8ead8a1f74e1e806ab0c3c1bc953c/users/50140e862037436581f713b8f1a38edb/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.370109 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:50.373353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4fd23-5e3d-430d-b9d2-81162e57001c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded8ead8a1f74e1e806ab0c3c1bc953c, user_id=50140e862037436581f713b8f1a38edb, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.376143 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4fd23-5e3d-430d-b9d2-81162e57001c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.388933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 993/1959] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:50 2022] PUT /identity/v3/projects/ded8ead8a1f74e1e806ab0c3c1bc953c/users/50140e862037436581f713b8f1a38edb/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:50.393046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6544c18-0572-43ae-affc-538ffe7e7c33 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.393341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6544c18-0572-43ae-affc-538ffe7e7c33 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.393618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6544c18-0572-43ae-affc-538ffe7e7c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.402265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691350bc-691d-4798-8cdf-726664d2aa94 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.411462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d6544c18-0572-43ae-affc-538ffe7e7c33 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:50.412714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d6544c18-0572-43ae-affc-538ffe7e7c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:50.434954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:50.458997 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177afe22-7b82-450a-8509-1b9a2bccda80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.459895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177afe22-7b82-450a-8509-1b9a2bccda80 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.460271 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177afe22-7b82-450a-8509-1b9a2bccda80 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.460628 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177afe22-7b82-450a-8509-1b9a2bccda80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.483376 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 994/1960] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:50.484794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6544c18-0572-43ae-affc-538ffe7e7c33 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.495384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6544c18-0572-43ae-affc-538ffe7e7c33 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.500339 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a49b2-5a3a-44a3-9542-2e03bd47dac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.501237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a49b2-5a3a-44a3-9542-2e03bd47dac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.501520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a49b2-5a3a-44a3-9542-2e03bd47dac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.501794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a49b2-5a3a-44a3-9542-2e03bd47dac9 admin admin] PATH_INFO: `/v3/projects/55726417bb7c4f04b274903e6b3a9c07/users/077828c6ab0046258b7e0c7365f6508b/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.517817 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a49b2-5a3a-44a3-9542-2e03bd47dac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55726417bb7c4f04b274903e6b3a9c07, user_id=077828c6ab0046258b7e0c7365f6508b, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.522608 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a49b2-5a3a-44a3-9542-2e03bd47dac9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.540580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177afe22-7b82-450a-8509-1b9a2bccda80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.542240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177afe22-7b82-450a-8509-1b9a2bccda80 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.548114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 995/1961] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:50 2022] PUT /identity/v3/projects/55726417bb7c4f04b274903e6b3a9c07/users/077828c6ab0046258b7e0c7365f6508b/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:50.548957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a49b2-5a3a-44a3-9542-2e03bd47dac9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.559358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 967/1962] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:50.560235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177afe22-7b82-450a-8509-1b9a2bccda80 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.570473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177afe22-7b82-450a-8509-1b9a2bccda80 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.674752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a49b2-5a3a-44a3-9542-2e03bd47dac9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.682669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108706e-67bc-4fd1-a1ad-2f42df1f06cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.683719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108706e-67bc-4fd1-a1ad-2f42df1f06cb admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.683919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cbfd2-0829-4ea5-b355-5357538efbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.684086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108706e-67bc-4fd1-a1ad-2f42df1f06cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.684305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108706e-67bc-4fd1-a1ad-2f42df1f06cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.685047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108706e-67bc-4fd1-a1ad-2f42df1f06cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.685407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cbfd2-0829-4ea5-b355-5357538efbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.685841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cbfd2-0829-4ea5-b355-5357538efbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.686306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cbfd2-0829-4ea5-b355-5357538efbc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.686815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108706e-67bc-4fd1-a1ad-2f42df1f06cb admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.687751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8cbfd2-0829-4ea5-b355-5357538efbc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:50.688387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8cbfd2-0829-4ea5-b355-5357538efbc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.690471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8cbfd2-0829-4ea5-b355-5357538efbc4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.703349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 996/1963] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:50.706135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971dce8-b1b0-431c-8121-aab1144ba3e1 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.706581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971dce8-b1b0-431c-8121-aab1144ba3e1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.707055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971dce8-b1b0-431c-8121-aab1144ba3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.711635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 968/1964] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:50.711635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cbfd2-0829-4ea5-b355-5357538efbc4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.723142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cbfd2-0829-4ea5-b355-5357538efbc4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.733717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94636ba6-b34d-411f-8cb0-2c668ed7d1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.736668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94636ba6-b34d-411f-8cb0-2c668ed7d1b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.737088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94636ba6-b34d-411f-8cb0-2c668ed7d1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.737990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94636ba6-b34d-411f-8cb0-2c668ed7d1b8 admin admin] PATH_INFO: `/v3/projects/ded8ead8a1f74e1e806ab0c3c1bc953c/users/50140e862037436581f713b8f1a38edb/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.741339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94636ba6-b34d-411f-8cb0-2c668ed7d1b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded8ead8a1f74e1e806ab0c3c1bc953c, user_id=50140e862037436581f713b8f1a38edb, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.745943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94636ba6-b34d-411f-8cb0-2c668ed7d1b8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.752687 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f971dce8-b1b0-431c-8121-aab1144ba3e1 None None] MFA Rules not processed for user `077828c6ab0046258b7e0c7365f6508b`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:50.752687 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f971dce8-b1b0-431c-8121-aab1144ba3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:50.780790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 969/1965] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:50 2022] PUT /identity/v3/projects/ded8ead8a1f74e1e806ab0c3c1bc953c/users/50140e862037436581f713b8f1a38edb/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:50.781799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94636ba6-b34d-411f-8cb0-2c668ed7d1b8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.797554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:50.854668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 997/1966] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:50 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:50.855441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971dce8-b1b0-431c-8121-aab1144ba3e1 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:50.855913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94636ba6-b34d-411f-8cb0-2c668ed7d1b8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.859152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f971dce8-b1b0-431c-8121-aab1144ba3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:50.861377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bd5db-9920-4347-9015-b6df65ef9a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.862165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bd5db-9920-4347-9015-b6df65ef9a26 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.862457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bd5db-9920-4347-9015-b6df65ef9a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.862737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bd5db-9920-4347-9015-b6df65ef9a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.863632 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bd5db-9920-4347-9015-b6df65ef9a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.865517 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bd5db-9920-4347-9015-b6df65ef9a26 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.873170 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971dce8-b1b0-431c-8121-aab1144ba3e1 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:50.877936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:50.878959 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.879673 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.879673 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] PATH_INFO: `/v3/users/90c6f467118d4ca9ad79f51231681995` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.881159 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c6f467118d4ca9ad79f51231681995)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:50.883171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:50.905625 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 970/1967] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:50 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:50.910311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e72115-d799-4c83-ad49-c067a3a0b6ac None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:50.910670 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e72115-d799-4c83-ad49-c067a3a0b6ac None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:50.912037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e72115-d799-4c83-ad49-c067a3a0b6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:50.922906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:50.951777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-72e72115-d799-4c83-ad49-c067a3a0b6ac None None] MFA Rules not processed for user `50140e862037436581f713b8f1a38edb`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:50.953975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-72e72115-d799-4c83-ad49-c067a3a0b6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:50.971119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c6f467118d4ca9ad79f51231681995'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:50.972173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c6f467118d4ca9ad79f51231681995'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:50.986921 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c6f467118d4ca9ad79f51231681995'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.016472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c6f467118d4ca9ad79f51231681995'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.034881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c6f467118d4ca9ad79f51231681995'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.041796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 998/1968] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:50 2022] DELETE /identity/v3/users/90c6f467118d4ca9ad79f51231681995 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:51.045901 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:51.047366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.069566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 971/1969] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:51.070698 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e72115-d799-4c83-ad49-c067a3a0b6ac None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:51.072597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-72e72115-d799-4c83-ad49-c067a3a0b6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.155448 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e72115-d799-4c83-ad49-c067a3a0b6ac None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:51.155596 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868d6b4-f395-4891-b52d-42db5a3a3c4f admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:51.160713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e43951-98a2-422e-8426-79704cc85368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:51.161238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fe05b-b566-47af-9ea8-1b3122112a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:51.161406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e43951-98a2-422e-8426-79704cc85368 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.161680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e43951-98a2-422e-8426-79704cc85368 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.161927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fe05b-b566-47af-9ea8-1b3122112a4a admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.162027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e43951-98a2-422e-8426-79704cc85368 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.162214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fe05b-b566-47af-9ea8-1b3122112a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.162498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fe05b-b566-47af-9ea8-1b3122112a4a admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.162654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e43951-98a2-422e-8426-79704cc85368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:51.163202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723fe05b-b566-47af-9ea8-1b3122112a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:51.164645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e43951-98a2-422e-8426-79704cc85368 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:51.165208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723fe05b-b566-47af-9ea8-1b3122112a4a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:51.191893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 972/1970] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:51 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:51.191893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fe05b-b566-47af-9ea8-1b3122112a4a admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:51.193663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-723fe05b-b566-47af-9ea8-1b3122112a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.194830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 999/1971] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:51 2022] POST /identity/v3/projects => generated 409 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:51.196155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3a065-a4d3-4475-8a21-216815fdbb15 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.196450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3a065-a4d3-4475-8a21-216815fdbb15 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.196732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3a065-a4d3-4475-8a21-216815fdbb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.209966 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fe05b-b566-47af-9ea8-1b3122112a4a admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:51.215433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-f5a3a065-a4d3-4475-8a21-216815fdbb15 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:51.216739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f5a3a065-a4d3-4475-8a21-216815fdbb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.218538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:51.267458 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:51.268381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.268663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.268938 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] PATH_INFO: `/v3/projects/dad61a7d4edd4c1cb22ef642af3acab6` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.270490 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad61a7d4edd4c1cb22ef642af3acab6)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:51.272468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:51.276968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1000/1972] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:51.278265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3a065-a4d3-4475-8a21-216815fdbb15 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:51.279703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f5a3a065-a4d3-4475-8a21-216815fdbb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.320853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:51.331198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad61a7d4edd4c1cb22ef642af3acab6', 'request_id': 'req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.342940 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad61a7d4edd4c1cb22ef642af3acab6', 'request_id': 'req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.351121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad61a7d4edd4c1cb22ef642af3acab6', 'request_id': 'req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.371454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] The token cache is being invalidate because project dad61a7d4edd4c1cb22ef642af3acab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:51.371680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.372454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.375024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 973/1973] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:51 2022] DELETE /identity/v3/projects/dad61a7d4edd4c1cb22ef642af3acab6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:51.375849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:51.378186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.393452 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3a065-a4d3-4475-8a21-216815fdbb15 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:51.476357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aea4b-2ae6-44e3-8213-209dcbc11f1a admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:51.492270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c9058-5ffc-466d-aa27-fb0863fcfd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:51.492669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fee68b-b6bf-4007-b6e1-e247e8a2471d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:51.493359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fee68b-b6bf-4007-b6e1-e247e8a2471d service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.493698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fee68b-b6bf-4007-b6e1-e247e8a2471d service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.494209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fee68b-b6bf-4007-b6e1-e247e8a2471d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.494617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c9058-5ffc-466d-aa27-fb0863fcfd25 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.495008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c9058-5ffc-466d-aa27-fb0863fcfd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.495380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c9058-5ffc-466d-aa27-fb0863fcfd25 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.498176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c9058-5ffc-466d-aa27-fb0863fcfd25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:51.498579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-83fee68b-b6bf-4007-b6e1-e247e8a2471d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.498683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c9058-5ffc-466d-aa27-fb0863fcfd25 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:51.515010 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 974/1974] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:51.519876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c9058-5ffc-466d-aa27-fb0863fcfd25 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:51.523219 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ad9c9058-5ffc-466d-aa27-fb0863fcfd25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.540163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c9058-5ffc-466d-aa27-fb0863fcfd25 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:51.549820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6176d-176a-4979-a90f-83bccd7eed6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:51.550796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6176d-176a-4979-a90f-83bccd7eed6d admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.551214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6176d-176a-4979-a90f-83bccd7eed6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.551728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6176d-176a-4979-a90f-83bccd7eed6d admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.552758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6176d-176a-4979-a90f-83bccd7eed6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:51.556526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6176d-176a-4979-a90f-83bccd7eed6d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:51.606981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 975/1975] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:51 2022] POST /identity/v3/users => generated 311 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:51.606981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6176d-176a-4979-a90f-83bccd7eed6d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:51.606981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-89f6176d-176a-4979-a90f-83bccd7eed6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.633887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:51.644343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fee68b-b6bf-4007-b6e1-e247e8a2471d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:51.648928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fee68b-b6bf-4007-b6e1-e247e8a2471d service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:51.677272 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1001/1976] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:51.679547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fee68b-b6bf-4007-b6e1-e247e8a2471d service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:51.681164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-83fee68b-b6bf-4007-b6e1-e247e8a2471d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.695570 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6176d-176a-4979-a90f-83bccd7eed6d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:51.697615 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fee68b-b6bf-4007-b6e1-e247e8a2471d service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:51.703397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591349c5-ade9-4a47-a9bf-5eb62d2bfdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:51.704373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591349c5-ade9-4a47-a9bf-5eb62d2bfdfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.704665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591349c5-ade9-4a47-a9bf-5eb62d2bfdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.704947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591349c5-ade9-4a47-a9bf-5eb62d2bfdfa admin admin] PATH_INFO: `/v3/users/433f06f7a1ed414ea06abac1abdf0cee` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.706202 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591349c5-ade9-4a47-a9bf-5eb62d2bfdfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=433f06f7a1ed414ea06abac1abdf0cee)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:51.707459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf688e4d-2f63-4db8-abff-3477772921a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:51.708169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf688e4d-2f63-4db8-abff-3477772921a2 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.708461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf688e4d-2f63-4db8-abff-3477772921a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.708562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591349c5-ade9-4a47-a9bf-5eb62d2bfdfa admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:51.710965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf688e4d-2f63-4db8-abff-3477772921a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.716666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bf688e4d-2f63-4db8-abff-3477772921a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.747626 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf688e4d-2f63-4db8-abff-3477772921a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:51.752686 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf688e4d-2f63-4db8-abff-3477772921a2 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:51.772679 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 976/1977] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:51.773819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf688e4d-2f63-4db8-abff-3477772921a2 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:51.792944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf688e4d-2f63-4db8-abff-3477772921a2 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:51.812135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa428c-d794-4d87-8d4e-774094febc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:51.812906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa428c-d794-4d87-8d4e-774094febc15 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.813196 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa428c-d794-4d87-8d4e-774094febc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.813822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa428c-d794-4d87-8d4e-774094febc15 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.815848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-591349c5-ade9-4a47-a9bf-5eb62d2bfdfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '433f06f7a1ed414ea06abac1abdf0cee'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.816827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa428c-d794-4d87-8d4e-774094febc15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:51.817552 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-591349c5-ade9-4a47-a9bf-5eb62d2bfdfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '433f06f7a1ed414ea06abac1abdf0cee'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.820408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa428c-d794-4d87-8d4e-774094febc15 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:51.839665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 977/1978] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:51.839665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa428c-d794-4d87-8d4e-774094febc15 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:51.840723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-591349c5-ade9-4a47-a9bf-5eb62d2bfdfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '433f06f7a1ed414ea06abac1abdf0cee'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.851613 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-baaa428c-d794-4d87-8d4e-774094febc15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.857111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-591349c5-ade9-4a47-a9bf-5eb62d2bfdfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '433f06f7a1ed414ea06abac1abdf0cee'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.879567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa428c-d794-4d87-8d4e-774094febc15 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:51.879879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-591349c5-ade9-4a47-a9bf-5eb62d2bfdfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '433f06f7a1ed414ea06abac1abdf0cee'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:51.891365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1002/1979] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:51 2022] DELETE /identity/v3/users/433f06f7a1ed414ea06abac1abdf0cee => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:51.895981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561c14e-7342-4f98-93ed-2db732eab968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:51.897334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af039300-f93e-4a49-9ac9-60a5b9d2af8d None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.897455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561c14e-7342-4f98-93ed-2db732eab968 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.897455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561c14e-7342-4f98-93ed-2db732eab968 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.897455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561c14e-7342-4f98-93ed-2db732eab968 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.898291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561c14e-7342-4f98-93ed-2db732eab968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:51.898703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af039300-f93e-4a49-9ac9-60a5b9d2af8d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.899232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af039300-f93e-4a49-9ac9-60a5b9d2af8d None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.899741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561c14e-7342-4f98-93ed-2db732eab968 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:51.903595 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1003/1980] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:49:51 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:49:51.905332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af039300-f93e-4a49-9ac9-60a5b9d2af8d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:51.908530 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-af039300-f93e-4a49-9ac9-60a5b9d2af8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:51.920384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:51.925962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 978/1981] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:51.928836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033a2d8-e1e2-425e-b0d8-b4274a0e90fb None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:51.929132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033a2d8-e1e2-425e-b0d8-b4274a0e90fb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:51.929419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033a2d8-e1e2-425e-b0d8-b4274a0e90fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:51.957725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b033a2d8-e1e2-425e-b0d8-b4274a0e90fb None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:51.958958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b033a2d8-e1e2-425e-b0d8-b4274a0e90fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:52.099670 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af039300-f93e-4a49-9ac9-60a5b9d2af8d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:52.101972 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 979/1982] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:49:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:52.102653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033a2d8-e1e2-425e-b0d8-b4274a0e90fb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:52.115525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033a2d8-e1e2-425e-b0d8-b4274a0e90fb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:52.129728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b571bb-d930-486c-9561-943244bfaa58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:52.130971 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b571bb-d930-486c-9561-943244bfaa58 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:52.131323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b571bb-d930-486c-9561-943244bfaa58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:52.131612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b571bb-d930-486c-9561-943244bfaa58 admin admin] PATH_INFO: `/v3/projects/d613c89fc1dc47f0b57e9d2a0c42e78d/users/aae60f23c96f46919402aed937020a79/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:52.146970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0dc30-db9f-487b-8ab5-e008e5f8ece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:52.148663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0dc30-db9f-487b-8ab5-e008e5f8ece5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:52.149169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0dc30-db9f-487b-8ab5-e008e5f8ece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:52.149664 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0dc30-db9f-487b-8ab5-e008e5f8ece5 admin admin] PATH_INFO: `/v3/projects/d877015ea5834afa9aabd4b681c50bd3/users/22d565f5221648c293152a609d7a9879/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:52.172740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0dc30-db9f-487b-8ab5-e008e5f8ece5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d877015ea5834afa9aabd4b681c50bd3, user_id=22d565f5221648c293152a609d7a9879, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:52.176611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0dc30-db9f-487b-8ab5-e008e5f8ece5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:52.179953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b571bb-d930-486c-9561-943244bfaa58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d613c89fc1dc47f0b57e9d2a0c42e78d, user_id=aae60f23c96f46919402aed937020a79, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:52.184557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b571bb-d930-486c-9561-943244bfaa58 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:52.209038 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1004/1983] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:51 2022] PUT /identity/v3/projects/d613c89fc1dc47f0b57e9d2a0c42e78d/users/aae60f23c96f46919402aed937020a79/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:52.219107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b571bb-d930-486c-9561-943244bfaa58 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:52.220350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a0b571bb-d930-486c-9561-943244bfaa58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:52.234672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 980/1984] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:52 2022] PUT /identity/v3/projects/d877015ea5834afa9aabd4b681c50bd3/users/22d565f5221648c293152a609d7a9879/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:52.236042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bbb02-6220-401b-8bc0-fd500bb836d8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:52.236642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bbb02-6220-401b-8bc0-fd500bb836d8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:52.236642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bbb02-6220-401b-8bc0-fd500bb836d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:52.283544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-fa7bbb02-6220-401b-8bc0-fd500bb836d8 None None] MFA Rules not processed for user `aae60f23c96f46919402aed937020a79`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:52.285652 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fa7bbb02-6220-401b-8bc0-fd500bb836d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:52.332756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b571bb-d930-486c-9561-943244bfaa58 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:52.360765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:52.375457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 981/1985] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:52.377207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bbb02-6220-401b-8bc0-fd500bb836d8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:52.456736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bbb02-6220-401b-8bc0-fd500bb836d8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:52.460566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ced6c6-c0c2-4f8b-a95f-954051f2bdfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:52.461266 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ced6c6-c0c2-4f8b-a95f-954051f2bdfa service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:52.461557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ced6c6-c0c2-4f8b-a95f-954051f2bdfa service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:52.461839 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ced6c6-c0c2-4f8b-a95f-954051f2bdfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:52.463684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4370832-84ba-4421-884b-6e46545ed025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:52.464593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4370832-84ba-4421-884b-6e46545ed025 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:52.465042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4370832-84ba-4421-884b-6e46545ed025 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:52.465481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4370832-84ba-4421-884b-6e46545ed025 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:52.466402 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4370832-84ba-4421-884b-6e46545ed025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:52.468658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4370832-84ba-4421-884b-6e46545ed025 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:52.471832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ced6c6-c0c2-4f8b-a95f-954051f2bdfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:52.473343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ced6c6-c0c2-4f8b-a95f-954051f2bdfa service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:52.483092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 982/1986] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:52.484339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4370832-84ba-4421-884b-6e46545ed025 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:52.487246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a4370832-84ba-4421-884b-6e46545ed025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:52.487742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1005/1987] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:52.488590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ced6c6-c0c2-4f8b-a95f-954051f2bdfa service nova] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:52.500343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ced6c6-c0c2-4f8b-a95f-954051f2bdfa service nova] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:52.509360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7efad0-2eea-4a2f-b20f-e735d37b1aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:52.509998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7efad0-2eea-4a2f-b20f-e735d37b1aec admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:52.510281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7efad0-2eea-4a2f-b20f-e735d37b1aec admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:52.510558 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7efad0-2eea-4a2f-b20f-e735d37b1aec admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:52.511250 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7efad0-2eea-4a2f-b20f-e735d37b1aec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:52.513549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7efad0-2eea-4a2f-b20f-e735d37b1aec admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:52.525822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:52.552936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1006/1988] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:52 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:52.554336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7efad0-2eea-4a2f-b20f-e735d37b1aec admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:52.556554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-be7efad0-2eea-4a2f-b20f-e735d37b1aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:52.599392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4370832-84ba-4421-884b-6e46545ed025 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:52.657262 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7efad0-2eea-4a2f-b20f-e735d37b1aec admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:52.661185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22f327-7a90-43ec-929b-2ac2980f65ed tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50140e862037436581f713b8f1a38edb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ded8ead8a1f74e1e806ab0c3c1bc953c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:52.662226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22f327-7a90-43ec-929b-2ac2980f65ed tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:52.662554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22f327-7a90-43ec-929b-2ac2980f65ed tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:52.662943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22f327-7a90-43ec-929b-2ac2980f65ed tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:52.664049 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22f327-7a90-43ec-929b-2ac2980f65ed tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:52.664289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22f327-7a90-43ec-929b-2ac2980f65ed tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:52.665830 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c22f327-7a90-43ec-929b-2ac2980f65ed tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:52.668997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:52.669902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:52.670177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:52.671220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] PATH_INFO: `/v3/projects/48cda030737e42d48c101e9b5315fa36` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:52.673521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48cda030737e42d48c101e9b5315fa36)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:52.675333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:52.690309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 983/1989] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:52.691486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22f327-7a90-43ec-929b-2ac2980f65ed tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:52.718274 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c22f327-7a90-43ec-929b-2ac2980f65ed tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:52.726348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cc174-cdd3-463d-a323-2c6dfe4df5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:52.727995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cc174-cdd3-463d-a323-2c6dfe4df5ae admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:52.728311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cc174-cdd3-463d-a323-2c6dfe4df5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:52.728745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cc174-cdd3-463d-a323-2c6dfe4df5ae admin admin] PATH_INFO: `/v3/projects/d877015ea5834afa9aabd4b681c50bd3/users/22d565f5221648c293152a609d7a9879/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:52.731441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cc174-cdd3-463d-a323-2c6dfe4df5ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d877015ea5834afa9aabd4b681c50bd3, user_id=22d565f5221648c293152a609d7a9879, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:52.734473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cc174-cdd3-463d-a323-2c6dfe4df5ae admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:52.751748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 984/1990] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:52 2022] PUT /identity/v3/projects/d877015ea5834afa9aabd4b681c50bd3/users/22d565f5221648c293152a609d7a9879/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:52.752576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cc174-cdd3-463d-a323-2c6dfe4df5ae admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:52.773434 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48cda030737e42d48c101e9b5315fa36', 'request_id': 'req-6d789b75-9ab2-4879-ad38-d6a50a99fb13'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:52.782889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48cda030737e42d48c101e9b5315fa36', 'request_id': 'req-6d789b75-9ab2-4879-ad38-d6a50a99fb13'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:52.791198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48cda030737e42d48c101e9b5315fa36', 'request_id': 'req-6d789b75-9ab2-4879-ad38-d6a50a99fb13'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:52.801901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:52.804246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] The token cache is being invalidate because project 48cda030737e42d48c101e9b5315fa36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:52.804554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:52.805294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:52.807087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1007/1991] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:52 2022] DELETE /identity/v3/projects/48cda030737e42d48c101e9b5315fa36 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:52.811482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:52.814339 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:52.848099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cc174-cdd3-463d-a323-2c6dfe4df5ae admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:52.894571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d789b75-9ab2-4879-ad38-d6a50a99fb13 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:52.899795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a71c6-4455-46bd-92bc-ddb5071dbc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:52.900743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a71c6-4455-46bd-92bc-ddb5071dbc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:52.901028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a71c6-4455-46bd-92bc-ddb5071dbc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:52.901309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a71c6-4455-46bd-92bc-ddb5071dbc7c admin admin] PATH_INFO: `/v3/projects/ae3410896d1747859c1ed4acddacad98` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:52.902556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a71c6-4455-46bd-92bc-ddb5071dbc7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3410896d1747859c1ed4acddacad98)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:52.904609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a71c6-4455-46bd-92bc-ddb5071dbc7c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:52.915421 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e853-923f-4038-a63f-7ee1f0f1c408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:52.916148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e853-923f-4038-a63f-7ee1f0f1c408 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:52.916436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e853-923f-4038-a63f-7ee1f0f1c408 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:52.916715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e853-923f-4038-a63f-7ee1f0f1c408 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:52.917409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9e853-923f-4038-a63f-7ee1f0f1c408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:52.919342 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9e853-923f-4038-a63f-7ee1f0f1c408 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:52.927223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:52.941338 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 985/1992] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:52 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:52.942434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e853-923f-4038-a63f-7ee1f0f1c408 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:52.947153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-26b9e853-923f-4038-a63f-7ee1f0f1c408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:52.960832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-591a71c6-4455-46bd-92bc-ddb5071dbc7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3410896d1747859c1ed4acddacad98', 'request_id': 'req-591a71c6-4455-46bd-92bc-ddb5071dbc7c'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:52.969078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-591a71c6-4455-46bd-92bc-ddb5071dbc7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3410896d1747859c1ed4acddacad98', 'request_id': 'req-591a71c6-4455-46bd-92bc-ddb5071dbc7c'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:52.975890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-591a71c6-4455-46bd-92bc-ddb5071dbc7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3410896d1747859c1ed4acddacad98', 'request_id': 'req-591a71c6-4455-46bd-92bc-ddb5071dbc7c'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:52.986485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-591a71c6-4455-46bd-92bc-ddb5071dbc7c admin admin] The token cache is being invalidate because project ae3410896d1747859c1ed4acddacad98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:52.986772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-591a71c6-4455-46bd-92bc-ddb5071dbc7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:52.987428 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-591a71c6-4455-46bd-92bc-ddb5071dbc7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:52.989310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1008/1993] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:52 2022] DELETE /identity/v3/projects/ae3410896d1747859c1ed4acddacad98 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:52.990345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8e827-f2f5-43f1-bf02-253c514c3100 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:52.990635 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8e827-f2f5-43f1-bf02-253c514c3100 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:52.990911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8e827-f2f5-43f1-bf02-253c514c3100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:53.017555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-0dd8e827-f2f5-43f1-bf02-253c514c3100 None None] MFA Rules not processed for user `22d565f5221648c293152a609d7a9879`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:53.018704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e853-923f-4038-a63f-7ee1f0f1c408 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:53.019213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0dd8e827-f2f5-43f1-bf02-253c514c3100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:53.079098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12df9a0-f8b7-4021-b1ac-42a9312a0a8c tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50140e862037436581f713b8f1a38edb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ded8ead8a1f74e1e806ab0c3c1bc953c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:53.079785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12df9a0-f8b7-4021-b1ac-42a9312a0a8c tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:53.080115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12df9a0-f8b7-4021-b1ac-42a9312a0a8c tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:53.080427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12df9a0-f8b7-4021-b1ac-42a9312a0a8c tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:53.081154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12df9a0-f8b7-4021-b1ac-42a9312a0a8c tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:53.083219 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12df9a0-f8b7-4021-b1ac-42a9312a0a8c tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:53.093168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:53.094368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1009/1994] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:52 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:53.098055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8e827-f2f5-43f1-bf02-253c514c3100 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:53.098055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0dd8e827-f2f5-43f1-bf02-253c514c3100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:53.116249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 986/1995] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:52 2022] POST /identity/v3/projects => generated 351 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:53.116249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12df9a0-f8b7-4021-b1ac-42a9312a0a8c tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:53.116249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a12df9a0-f8b7-4021-b1ac-42a9312a0a8c tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:53.163905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8e827-f2f5-43f1-bf02-253c514c3100 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:53.180373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12df9a0-f8b7-4021-b1ac-42a9312a0a8c tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:53.185272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff5e25-0c63-47ac-b5ad-b098423694bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:53.185955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff5e25-0c63-47ac-b5ad-b098423694bd admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:53.186241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff5e25-0c63-47ac-b5ad-b098423694bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:53.186544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff5e25-0c63-47ac-b5ad-b098423694bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:53.187278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff5e25-0c63-47ac-b5ad-b098423694bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:53.188962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff5e25-0c63-47ac-b5ad-b098423694bd admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:53.198733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 987/1996] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:53.200037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937840d-ccba-4a47-8880-504d7198b207 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:53.200327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937840d-ccba-4a47-8880-504d7198b207 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:53.200604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937840d-ccba-4a47-8880-504d7198b207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:53.205749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:53.219137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3937840d-ccba-4a47-8880-504d7198b207 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:53.220388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3937840d-ccba-4a47-8880-504d7198b207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:53.224330 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:53.225221 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:53.225499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:53.225775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] PATH_INFO: `/v3/users/c28c03459b4648d3bc763b5ca8157811` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:53.226984 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c28c03459b4648d3bc763b5ca8157811)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:53.228839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 988/1997] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:53.229005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:53.229423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937840d-ccba-4a47-8880-504d7198b207 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:53.230635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3937840d-ccba-4a47-8880-504d7198b207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:53.280975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c28c03459b4648d3bc763b5ca8157811'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:53.281639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c28c03459b4648d3bc763b5ca8157811'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:53.289121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937840d-ccba-4a47-8880-504d7198b207 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:53.290351 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c28c03459b4648d3bc763b5ca8157811'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:53.292931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a705e-5edc-4dfd-9b78-199aadf92ce2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:53.293596 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a705e-5edc-4dfd-9b78-199aadf92ce2 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:53.293884 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a705e-5edc-4dfd-9b78-199aadf92ce2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:53.294162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a705e-5edc-4dfd-9b78-199aadf92ce2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:53.296034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7f9a705e-5edc-4dfd-9b78-199aadf92ce2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:53.297599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c28c03459b4648d3bc763b5ca8157811'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:53.309496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c28c03459b4648d3bc763b5ca8157811'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:53.315553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1010/1998] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:53 2022] DELETE /identity/v3/users/c28c03459b4648d3bc763b5ca8157811 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:53.316507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:53.317894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:53.371229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a705e-5edc-4dfd-9b78-199aadf92ce2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:53.373381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a705e-5edc-4dfd-9b78-199aadf92ce2 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:53.387427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 989/1999] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:53 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:53.388725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a705e-5edc-4dfd-9b78-199aadf92ce2 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:53.390417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7f9a705e-5edc-4dfd-9b78-199aadf92ce2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:53.405106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a705e-5edc-4dfd-9b78-199aadf92ce2 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:53.417285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9edfe9-aff8-4183-b0e6-d1650cfe89c9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:53.428825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef21838-227a-4c64-b1cc-5ca14c141ecf tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50140e862037436581f713b8f1a38edb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ded8ead8a1f74e1e806ab0c3c1bc953c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:53.432267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef21838-227a-4c64-b1cc-5ca14c141ecf tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:53.432846 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef21838-227a-4c64-b1cc-5ca14c141ecf tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:53.433362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef21838-227a-4c64-b1cc-5ca14c141ecf tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:53.434320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef21838-227a-4c64-b1cc-5ca14c141ecf tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:53.435057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef21838-227a-4c64-b1cc-5ca14c141ecf tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:53.438057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef21838-227a-4c64-b1cc-5ca14c141ecf tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:53.466359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1011/2000] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:53.467431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d209f5-4a72-4229-8544-9ba589a5a142 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:53.467870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d209f5-4a72-4229-8544-9ba589a5a142 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:53.467870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d209f5-4a72-4229-8544-9ba589a5a142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:53.475683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:53.514261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-b5d209f5-4a72-4229-8544-9ba589a5a142 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:53.515512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b5d209f5-4a72-4229-8544-9ba589a5a142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:53.564127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cc960-65c8-48db-99cc-dd8821402c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:53.567503 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cc960-65c8-48db-99cc-dd8821402c65 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:53.567944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cc960-65c8-48db-99cc-dd8821402c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:53.568796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1cc960-65c8-48db-99cc-dd8821402c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:53.571827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1cc960-65c8-48db-99cc-dd8821402c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:53.577051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1cc960-65c8-48db-99cc-dd8821402c65 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:53.591894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:53.610416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 990/2001] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:53.610416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cc960-65c8-48db-99cc-dd8821402c65 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:53.614350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1012/2002] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:53.614350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d209f5-4a72-4229-8544-9ba589a5a142 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:53.626388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1cc960-65c8-48db-99cc-dd8821402c65 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:53.634710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50140e862037436581f713b8f1a38edb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ded8ead8a1f74e1e806ab0c3c1bc953c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:53.636311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:53.636755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:53.637388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] PATH_INFO: `/v3/projects/b268bebc8da84e36a2ceee466aadc12d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:53.639419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b268bebc8da84e36a2ceee466aadc12d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:53.641617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:53.655519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d209f5-4a72-4229-8544-9ba589a5a142 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:53.671502 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476644e3-b888-4d8b-a351-69f0451c5d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:53.672525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476644e3-b888-4d8b-a351-69f0451c5d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:53.672790 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476644e3-b888-4d8b-a351-69f0451c5d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:53.673065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476644e3-b888-4d8b-a351-69f0451c5d38 admin admin] PATH_INFO: `/v3/projects/a04facd58fa9499bbf83831df037b3d4/users/54d68ccbc595499a97bd1fe422adcb20/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:53.707521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476644e3-b888-4d8b-a351-69f0451c5d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a04facd58fa9499bbf83831df037b3d4, user_id=54d68ccbc595499a97bd1fe422adcb20, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:53.711433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476644e3-b888-4d8b-a351-69f0451c5d38 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:53.730713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1013/2003] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:53 2022] PUT /identity/v3/projects/a04facd58fa9499bbf83831df037b3d4/users/54d68ccbc595499a97bd1fe422adcb20/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:53.732182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476644e3-b888-4d8b-a351-69f0451c5d38 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:53.789452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b268bebc8da84e36a2ceee466aadc12d', 'request_id': 'req-9640b290-70b5-4d0c-8b02-ebff08318865'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:53.818862 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b268bebc8da84e36a2ceee466aadc12d', 'request_id': 'req-9640b290-70b5-4d0c-8b02-ebff08318865'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:53.833249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b268bebc8da84e36a2ceee466aadc12d', 'request_id': 'req-9640b290-70b5-4d0c-8b02-ebff08318865'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:53.859536 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] The token cache is being invalidate because project b268bebc8da84e36a2ceee466aadc12d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:53.859879 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:53.860868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476644e3-b888-4d8b-a351-69f0451c5d38 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:53.873171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:53.877409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 991/2004] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:53 2022] DELETE /identity/v3/projects/b268bebc8da84e36a2ceee466aadc12d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:53.878577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:53.883796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.000207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57b25c-b91e-4f5e-9ba8-58cf24b5f001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:54.000921 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57b25c-b91e-4f5e-9ba8-58cf24b5f001 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.001208 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57b25c-b91e-4f5e-9ba8-58cf24b5f001 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.001486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57b25c-b91e-4f5e-9ba8-58cf24b5f001 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.002223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57b25c-b91e-4f5e-9ba8-58cf24b5f001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:54.002533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57b25c-b91e-4f5e-9ba8-58cf24b5f001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:54.004556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57b25c-b91e-4f5e-9ba8-58cf24b5f001 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:54.033090 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1014/2005] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:54.034585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d086b-1110-4845-bd92-a8fb24bf3615 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.034930 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d086b-1110-4845-bd92-a8fb24bf3615 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.035210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d086b-1110-4845-bd92-a8fb24bf3615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.051690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640b290-70b5-4d0c-8b02-ebff08318865 tempest-VolumeQuotaClassesTest-1560426893 tempest-VolumeQuotaClassesTest-1560426893-project-admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:54.056234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bc462-d1c6-43fd-9a4b-a21a0010ec24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:54.057088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bc462-d1c6-43fd-9a4b-a21a0010ec24 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.057815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bc462-d1c6-43fd-9a4b-a21a0010ec24 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.058094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bc462-d1c6-43fd-9a4b-a21a0010ec24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.061169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-058bc462-d1c6-43fd-9a4b-a21a0010ec24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.068274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-663d086b-1110-4845-bd92-a8fb24bf3615 None None] MFA Rules not processed for user `54d68ccbc595499a97bd1fe422adcb20`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:54.069731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-663d086b-1110-4845-bd92-a8fb24bf3615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.139272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bc462-d1c6-43fd-9a4b-a21a0010ec24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:54.141179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bc462-d1c6-43fd-9a4b-a21a0010ec24 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:54.158464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 992/2006] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:53 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:54.162007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bc462-d1c6-43fd-9a4b-a21a0010ec24 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:54.169275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-058bc462-d1c6-43fd-9a4b-a21a0010ec24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.173332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1015/2007] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:54.174462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d086b-1110-4845-bd92-a8fb24bf3615 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:54.177658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-663d086b-1110-4845-bd92-a8fb24bf3615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.205155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:54.213756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:54.215786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bc462-d1c6-43fd-9a4b-a21a0010ec24 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:54.225562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d086b-1110-4845-bd92-a8fb24bf3615 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:54.227461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:54.228933 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.229441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.229780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] PATH_INFO: `/v3/projects/6ead544a8a444568b7b683d6a1977c27` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.238392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ead544a8a444568b7b683d6a1977c27)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:54.244244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:54.250659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a3879-8241-43dc-ae55-93b9a0461df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:54.251362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a3879-8241-43dc-ae55-93b9a0461df9 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.254461 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a3879-8241-43dc-ae55-93b9a0461df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.256703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a3879-8241-43dc-ae55-93b9a0461df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.260620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a3879-8241-43dc-ae55-93b9a0461df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:54.268144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a3879-8241-43dc-ae55-93b9a0461df9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:54.343846 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ead544a8a444568b7b683d6a1977c27', 'request_id': 'req-d46602a7-0e0b-42de-9feb-f73f6303b8e9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:54.350631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1016/2008] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:54 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:54.355474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a3879-8241-43dc-ae55-93b9a0461df9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:54.358360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ead544a8a444568b7b683d6a1977c27', 'request_id': 'req-d46602a7-0e0b-42de-9feb-f73f6303b8e9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:54.359466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-da4a3879-8241-43dc-ae55-93b9a0461df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.372718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ead544a8a444568b7b683d6a1977c27', 'request_id': 'req-d46602a7-0e0b-42de-9feb-f73f6303b8e9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:54.402476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] The token cache is being invalidate because project 6ead544a8a444568b7b683d6a1977c27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:54.402779 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:54.404038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:54.407019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 993/2009] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:54 2022] DELETE /identity/v3/projects/6ead544a8a444568b7b683d6a1977c27 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:54.407917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:54.412225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.517632 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46602a7-0e0b-42de-9feb-f73f6303b8e9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:54.523250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bfa90-fe93-4ba3-970d-6313d97bc21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:54.523480 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a3879-8241-43dc-ae55-93b9a0461df9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:54.524467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bfa90-fe93-4ba3-970d-6313d97bc21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.525050 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bfa90-fe93-4ba3-970d-6313d97bc21e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.525339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bfa90-fe93-4ba3-970d-6313d97bc21e admin admin] PATH_INFO: `/v3/users/809518ef65fb410ea8b522b13798a87c` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.526782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bfa90-fe93-4ba3-970d-6313d97bc21e admin admin] RBAC: Authorizing `identity:delete_user(user_id=809518ef65fb410ea8b522b13798a87c)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:54.528987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bfa90-fe93-4ba3-970d-6313d97bc21e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:54.530361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60d7aa-0b0e-49e1-8976-78344ef0dacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:54.531104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60d7aa-0b0e-49e1-8976-78344ef0dacd admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.531394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60d7aa-0b0e-49e1-8976-78344ef0dacd admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.532061 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60d7aa-0b0e-49e1-8976-78344ef0dacd admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.532503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60d7aa-0b0e-49e1-8976-78344ef0dacd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:54.534474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60d7aa-0b0e-49e1-8976-78344ef0dacd admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:54.574278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1017/2010] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:54 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:54.575196 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60d7aa-0b0e-49e1-8976-78344ef0dacd admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:54.577220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8e60d7aa-0b0e-49e1-8976-78344ef0dacd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.594411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3e6bfa90-fe93-4ba3-970d-6313d97bc21e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '809518ef65fb410ea8b522b13798a87c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:54.605351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3e6bfa90-fe93-4ba3-970d-6313d97bc21e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '809518ef65fb410ea8b522b13798a87c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:54.606117 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3e6bfa90-fe93-4ba3-970d-6313d97bc21e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '809518ef65fb410ea8b522b13798a87c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:54.615658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3e6bfa90-fe93-4ba3-970d-6313d97bc21e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '809518ef65fb410ea8b522b13798a87c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:54.644881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3e6bfa90-fe93-4ba3-970d-6313d97bc21e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '809518ef65fb410ea8b522b13798a87c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:54.674805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 994/2011] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:54 2022] DELETE /identity/v3/users/809518ef65fb410ea8b522b13798a87c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:54.676692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7bc02-c090-4f71-9876-2d73baac6737 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.677051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7bc02-c090-4f71-9876-2d73baac6737 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.677478 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7bc02-c090-4f71-9876-2d73baac6737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.681946 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60d7aa-0b0e-49e1-8976-78344ef0dacd admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:54.718312 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:54.731311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-31a7bc02-c090-4f71-9876-2d73baac6737 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:54.736645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-31a7bc02-c090-4f71-9876-2d73baac6737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.779207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981011aa-ba07-439f-93f0-862bbf88710d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:54.779952 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981011aa-ba07-439f-93f0-862bbf88710d admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.780269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981011aa-ba07-439f-93f0-862bbf88710d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.780546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981011aa-ba07-439f-93f0-862bbf88710d admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.781247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981011aa-ba07-439f-93f0-862bbf88710d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:54.783496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981011aa-ba07-439f-93f0-862bbf88710d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:54.809986 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1018/2012] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:54 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:54.810961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981011aa-ba07-439f-93f0-862bbf88710d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:54.812696 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-981011aa-ba07-439f-93f0-862bbf88710d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.820203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:54.826737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981011aa-ba07-439f-93f0-862bbf88710d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:54.828033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 995/2013] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:54.828033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd874c99-a5f5-4862-af45-45fc0326af8a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.828033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd874c99-a5f5-4862-af45-45fc0326af8a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.828033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd874c99-a5f5-4862-af45-45fc0326af8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.840146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21daa2-6d47-4f01-881a-51929e16ba26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:54.841140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21daa2-6d47-4f01-881a-51929e16ba26 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.841609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21daa2-6d47-4f01-881a-51929e16ba26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.842077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21daa2-6d47-4f01-881a-51929e16ba26 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.842948 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21daa2-6d47-4f01-881a-51929e16ba26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:54.845303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21daa2-6d47-4f01-881a-51929e16ba26 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:54.857163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dd874c99-a5f5-4862-af45-45fc0326af8a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:54.858593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dd874c99-a5f5-4862-af45-45fc0326af8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.869400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 996/2014] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:54.870509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd874c99-a5f5-4862-af45-45fc0326af8a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:54.871972 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dd874c99-a5f5-4862-af45-45fc0326af8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.880663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1019/2015] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:54 2022] POST /identity/v3/users => generated 295 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:54.882418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39182169-fd46-4013-ba48-465b3aa325ce None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.883258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39182169-fd46-4013-ba48-465b3aa325ce None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.884958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39182169-fd46-4013-ba48-465b3aa325ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.890540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd874c99-a5f5-4862-af45-45fc0326af8a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:54.898539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42605a-0ced-4be1-bd0c-9b006cac0af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:54.898947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42605a-0ced-4be1-bd0c-9b006cac0af2 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.899229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42605a-0ced-4be1-bd0c-9b006cac0af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.899497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42605a-0ced-4be1-bd0c-9b006cac0af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.900324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42605a-0ced-4be1-bd0c-9b006cac0af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:54.902680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42605a-0ced-4be1-bd0c-9b006cac0af2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:54.915296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-39182169-fd46-4013-ba48-465b3aa325ce None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:54.915296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-39182169-fd46-4013-ba48-465b3aa325ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.915563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 997/2016] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:54.917749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1020/2017] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:54.918631 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42605a-0ced-4be1-bd0c-9b006cac0af2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:54.921614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39182169-fd46-4013-ba48-465b3aa325ce None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:54.923174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-39182169-fd46-4013-ba48-465b3aa325ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:54.930192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42605a-0ced-4be1-bd0c-9b006cac0af2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:54.935785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd47c59-89fe-48b7-b6b0-5f42bf95d94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:54.936696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd47c59-89fe-48b7-b6b0-5f42bf95d94d admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.936958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd47c59-89fe-48b7-b6b0-5f42bf95d94d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.937234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd47c59-89fe-48b7-b6b0-5f42bf95d94d admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.938060 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd47c59-89fe-48b7-b6b0-5f42bf95d94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:54.939962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd47c59-89fe-48b7-b6b0-5f42bf95d94d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:54.953686 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 998/2018] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:54.954620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd47c59-89fe-48b7-b6b0-5f42bf95d94d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:54.965144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd47c59-89fe-48b7-b6b0-5f42bf95d94d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:54.972448 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235eca1-4eb2-41f4-8b93-b40d332ecd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:54.973302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235eca1-4eb2-41f4-8b93-b40d332ecd44 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:54.973557 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235eca1-4eb2-41f4-8b93-b40d332ecd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:54.973834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235eca1-4eb2-41f4-8b93-b40d332ecd44 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:54.974582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235eca1-4eb2-41f4-8b93-b40d332ecd44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:54.976419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235eca1-4eb2-41f4-8b93-b40d332ecd44 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:54.991757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 999/2019] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:54.992788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235eca1-4eb2-41f4-8b93-b40d332ecd44 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.004953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235eca1-4eb2-41f4-8b93-b40d332ecd44 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.005833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39182169-fd46-4013-ba48-465b3aa325ce None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.009751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13728b98-55ac-4bb3-8220-848255700b34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.010426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13728b98-55ac-4bb3-8220-848255700b34 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.010711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13728b98-55ac-4bb3-8220-848255700b34 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.010829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad583c-fb07-4ac9-878f-e620fafe79ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.011009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13728b98-55ac-4bb3-8220-848255700b34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.011539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad583c-fb07-4ac9-878f-e620fafe79ba admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.011777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad583c-fb07-4ac9-878f-e620fafe79ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.012198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad583c-fb07-4ac9-878f-e620fafe79ba admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.012941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad583c-fb07-4ac9-878f-e620fafe79ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:55.013300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad583c-fb07-4ac9-878f-e620fafe79ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.016248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad583c-fb07-4ac9-878f-e620fafe79ba admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.035522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1000/2020] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:49:55.036545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad583c-fb07-4ac9-878f-e620fafe79ba admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.042883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13728b98-55ac-4bb3-8220-848255700b34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.045181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13728b98-55ac-4bb3-8220-848255700b34 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.050545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad583c-fb07-4ac9-878f-e620fafe79ba admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.062561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedb159-831c-4a90-b743-5054c77b4414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.065774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedb159-831c-4a90-b743-5054c77b4414 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.066327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedb159-831c-4a90-b743-5054c77b4414 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.066900 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedb159-831c-4a90-b743-5054c77b4414 admin admin] PATH_INFO: `/v3/projects/8b4d2c3e26b04e1d9dde9a63623c4ae8/users/c1396f966f51467ebb436112d538dcae/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.079287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1021/2021] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:55.080217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13728b98-55ac-4bb3-8220-848255700b34 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.089484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedb159-831c-4a90-b743-5054c77b4414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4d2c3e26b04e1d9dde9a63623c4ae8, user_id=c1396f966f51467ebb436112d538dcae, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.094427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedb159-831c-4a90-b743-5054c77b4414 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.098305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13728b98-55ac-4bb3-8220-848255700b34 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.110101 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4c1fb-163a-4cb7-817e-a0b342d20a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.111030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c1fb-163a-4cb7-817e-a0b342d20a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.111313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c1fb-163a-4cb7-817e-a0b342d20a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.111593 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c1fb-163a-4cb7-817e-a0b342d20a23 admin admin] PATH_INFO: `/v3/projects/e0fd252826f243cd9430ba1a4cee95c5/users/7c767bec3f4c4c9c8698afce8e40fd19/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.117583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1001/2022] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:55 2022] PUT /identity/v3/projects/8b4d2c3e26b04e1d9dde9a63623c4ae8/users/c1396f966f51467ebb436112d538dcae/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:55.119200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedb159-831c-4a90-b743-5054c77b4414 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.135703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4c1fb-163a-4cb7-817e-a0b342d20a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fd252826f243cd9430ba1a4cee95c5, user_id=7c767bec3f4c4c9c8698afce8e40fd19, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.138591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4c1fb-163a-4cb7-817e-a0b342d20a23 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.166274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1022/2023] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:55 2022] PUT /identity/v3/projects/e0fd252826f243cd9430ba1a4cee95c5/users/7c767bec3f4c4c9c8698afce8e40fd19/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:55.167338 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4c1fb-163a-4cb7-817e-a0b342d20a23 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.240177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4c1fb-163a-4cb7-817e-a0b342d20a23 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.243731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedb159-831c-4a90-b743-5054c77b4414 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.245314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98235a00-e75d-4d70-8fbf-c8b0ea86aacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.246082 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98235a00-e75d-4d70-8fbf-c8b0ea86aacc admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.246224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98235a00-e75d-4d70-8fbf-c8b0ea86aacc admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.246502 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98235a00-e75d-4d70-8fbf-c8b0ea86aacc admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.247452 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98235a00-e75d-4d70-8fbf-c8b0ea86aacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.249246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98235a00-e75d-4d70-8fbf-c8b0ea86aacc admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.260602 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f9759-6bb3-481f-a40d-ab5d37b238a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.260772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1023/2024] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:55.261374 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f9759-6bb3-481f-a40d-ab5d37b238a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.261728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60a81c-e2ed-4145-a792-92921093b0e4 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.262020 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60a81c-e2ed-4145-a792-92921093b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.262293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60a81c-e2ed-4145-a792-92921093b0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.262503 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f9759-6bb3-481f-a40d-ab5d37b238a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.263522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f9759-6bb3-481f-a40d-ab5d37b238a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.264379 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f9759-6bb3-481f-a40d-ab5d37b238a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.268101 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f9759-6bb3-481f-a40d-ab5d37b238a3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.281253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-6c60a81c-e2ed-4145-a792-92921093b0e4 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:55.282759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6c60a81c-e2ed-4145-a792-92921093b0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:55.297686 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1024/2025] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:55.298812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60a81c-e2ed-4145-a792-92921093b0e4 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.315155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60a81c-e2ed-4145-a792-92921093b0e4 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.320440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf85049-564e-4f85-9050-fdf97a1293ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.321347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf85049-564e-4f85-9050-fdf97a1293ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.321625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf85049-564e-4f85-9050-fdf97a1293ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.321898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf85049-564e-4f85-9050-fdf97a1293ac admin admin] PATH_INFO: `/v3/projects/8b4d2c3e26b04e1d9dde9a63623c4ae8/users/c1396f966f51467ebb436112d538dcae/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.323864 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf85049-564e-4f85-9050-fdf97a1293ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4d2c3e26b04e1d9dde9a63623c4ae8, user_id=c1396f966f51467ebb436112d538dcae, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.326666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf85049-564e-4f85-9050-fdf97a1293ac admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.331276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1002/2026] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:55 2022] POST /identity/v3/projects => generated 387 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:55.333265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f9759-6bb3-481f-a40d-ab5d37b238a3 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.340854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1025/2027] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:55 2022] PUT /identity/v3/projects/8b4d2c3e26b04e1d9dde9a63623c4ae8/users/c1396f966f51467ebb436112d538dcae/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:55.341332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf85049-564e-4f85-9050-fdf97a1293ac admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.378105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:55.381245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:55.416326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf85049-564e-4f85-9050-fdf97a1293ac admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.421637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faa906-489e-4ada-ae1c-31745b7ed876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.422576 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faa906-489e-4ada-ae1c-31745b7ed876 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.423008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faa906-489e-4ada-ae1c-31745b7ed876 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.423419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faa906-489e-4ada-ae1c-31745b7ed876 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.424298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faa906-489e-4ada-ae1c-31745b7ed876 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.426646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faa906-489e-4ada-ae1c-31745b7ed876 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.437701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f9759-6bb3-481f-a40d-ab5d37b238a3 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.465387 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1026/2028] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:55 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:55.467116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35756ae3-c13e-4496-88d9-7cb5de825956 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.467903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35756ae3-c13e-4496-88d9-7cb5de825956 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.469208 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35756ae3-c13e-4496-88d9-7cb5de825956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.529369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-35756ae3-c13e-4496-88d9-7cb5de825956 None None] MFA Rules not processed for user `c1396f966f51467ebb436112d538dcae`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:55.531296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-35756ae3-c13e-4496-88d9-7cb5de825956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:55.611690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67b85a-147e-415d-81e3-49957ee58f4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.612692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67b85a-147e-415d-81e3-49957ee58f4a service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.613317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67b85a-147e-415d-81e3-49957ee58f4a service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.613956 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67b85a-147e-415d-81e3-49957ee58f4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.633832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67b85a-147e-415d-81e3-49957ee58f4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.636986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67b85a-147e-415d-81e3-49957ee58f4a service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.655672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1003/2029] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:55.658092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67b85a-147e-415d-81e3-49957ee58f4a service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.674477 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1027/2030] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:55.680293 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67b85a-147e-415d-81e3-49957ee58f4a service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.696953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35756ae3-c13e-4496-88d9-7cb5de825956 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.698538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-35756ae3-c13e-4496-88d9-7cb5de825956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:55.701601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e6f67-e8ba-4366-9a59-0b2c71f2fec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.702400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e6f67-e8ba-4366-9a59-0b2c71f2fec1 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.702782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e6f67-e8ba-4366-9a59-0b2c71f2fec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.703827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e6f67-e8ba-4366-9a59-0b2c71f2fec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.705360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e6f67-e8ba-4366-9a59-0b2c71f2fec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.710986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e6f67-e8ba-4366-9a59-0b2c71f2fec1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.760099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1004/2031] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:55.765196 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e6f67-e8ba-4366-9a59-0b2c71f2fec1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.798287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e6f67-e8ba-4366-9a59-0b2c71f2fec1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.819803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbe4a8-e67d-4017-9ed8-6f96bdb0cc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.820964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbe4a8-e67d-4017-9ed8-6f96bdb0cc37 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.824114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbe4a8-e67d-4017-9ed8-6f96bdb0cc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.826425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbe4a8-e67d-4017-9ed8-6f96bdb0cc37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.827250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbe4a8-e67d-4017-9ed8-6f96bdb0cc37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.841965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbe4a8-e67d-4017-9ed8-6f96bdb0cc37 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.847241 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35756ae3-c13e-4496-88d9-7cb5de825956 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.851826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86903a-b0c1-490f-921f-ea9d9fb53fdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.852845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86903a-b0c1-490f-921f-ea9d9fb53fdb service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.853369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86903a-b0c1-490f-921f-ea9d9fb53fdb service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.853749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86903a-b0c1-490f-921f-ea9d9fb53fdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.873602 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86903a-b0c1-490f-921f-ea9d9fb53fdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.876259 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86903a-b0c1-490f-921f-ea9d9fb53fdb service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.879820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1005/2032] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:55.881356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbe4a8-e67d-4017-9ed8-6f96bdb0cc37 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.897084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1028/2033] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:55.906906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbe4a8-e67d-4017-9ed8-6f96bdb0cc37 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.914102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86903a-b0c1-490f-921f-ea9d9fb53fdb service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:55.916022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ca86903a-b0c1-490f-921f-ea9d9fb53fdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:55.925295 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a2d24-ebde-44c1-b8ec-246cfd57638e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.930602 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a2d24-ebde-44c1-b8ec-246cfd57638e admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.930959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a2d24-ebde-44c1-b8ec-246cfd57638e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.931253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a2d24-ebde-44c1-b8ec-246cfd57638e admin admin] PATH_INFO: `/v3/projects/e6759753399645b99c721a54ca969214/users/8fa2d0c8c4124774be30fa97e36c94dd/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.937314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86903a-b0c1-490f-921f-ea9d9fb53fdb service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:55.948198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:55.949686 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:55.950080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:55.950488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e admin admin] PATH_INFO: `/v3/projects/946175c6149b46b1838d8ebb93efba66` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:55.952186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e admin admin] RBAC: Authorizing `identity:delete_project(project_id=946175c6149b46b1838d8ebb93efba66)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.954603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:55.988642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a2d24-ebde-44c1-b8ec-246cfd57638e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6759753399645b99c721a54ca969214, user_id=8fa2d0c8c4124774be30fa97e36c94dd, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:55.998673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a2d24-ebde-44c1-b8ec-246cfd57638e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:56.047595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1006/2034] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:55 2022] PUT /identity/v3/projects/e6759753399645b99c721a54ca969214/users/8fa2d0c8c4124774be30fa97e36c94dd/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:56.047595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a2d24-ebde-44c1-b8ec-246cfd57638e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:56.081612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:56.088221 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '946175c6149b46b1838d8ebb93efba66', 'request_id': 'req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:56.133410 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '946175c6149b46b1838d8ebb93efba66', 'request_id': 'req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:56.149985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '946175c6149b46b1838d8ebb93efba66', 'request_id': 'req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:56.184401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e admin admin] The token cache is being invalidate because project 946175c6149b46b1838d8ebb93efba66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:56.185052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a2d24-ebde-44c1-b8ec-246cfd57638e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:56.185890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:56.186904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-c5ba51c1-4c9e-43a3-9e7c-563bdf07c18e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:56.189658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1029/2035] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:55 2022] DELETE /identity/v3/projects/946175c6149b46b1838d8ebb93efba66 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:49:56.191444 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de54f91-7703-4097-a813-c2af30f4f6ac None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:56.191942 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de54f91-7703-4097-a813-c2af30f4f6ac None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:56.192693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de54f91-7703-4097-a813-c2af30f4f6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:56.237415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8de54f91-7703-4097-a813-c2af30f4f6ac None None] MFA Rules not processed for user `8fa2d0c8c4124774be30fa97e36c94dd`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:56.240193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8de54f91-7703-4097-a813-c2af30f4f6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:56.330752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1030/2036] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:56.331748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de54f91-7703-4097-a813-c2af30f4f6ac None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:56.333103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8de54f91-7703-4097-a813-c2af30f4f6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:56.393695 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:56.439335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c4000-b91f-4d8b-bc77-fec7acef2ca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:56.440437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c4000-b91f-4d8b-bc77-fec7acef2ca3 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:56.440437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c4000-b91f-4d8b-bc77-fec7acef2ca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:56.440742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c4000-b91f-4d8b-bc77-fec7acef2ca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:56.451330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-824c4000-b91f-4d8b-bc77-fec7acef2ca3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:56.493355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de54f91-7703-4097-a813-c2af30f4f6ac None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:56.502266 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:56.503228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:56.503603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:56.503882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] PATH_INFO: `/v3/users/50140e862037436581f713b8f1a38edb` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:56.505194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] RBAC: Authorizing `identity:delete_user(user_id=50140e862037436581f713b8f1a38edb)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:56.507224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:56.539030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c4000-b91f-4d8b-bc77-fec7acef2ca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:56.541005 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c4000-b91f-4d8b-bc77-fec7acef2ca3 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:56.564913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1007/2037] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:56.565082 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50140e862037436581f713b8f1a38edb'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:56.565711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c4000-b91f-4d8b-bc77-fec7acef2ca3 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:56.567342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50140e862037436581f713b8f1a38edb'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:56.569230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-824c4000-b91f-4d8b-bc77-fec7acef2ca3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:56.581551 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50140e862037436581f713b8f1a38edb'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:56.598798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50140e862037436581f713b8f1a38edb'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:56.602945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:56.604560 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c4000-b91f-4d8b-bc77-fec7acef2ca3 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:56.614116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d6851-2250-4cfd-969d-f8c9ad9f9ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:56.614810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d6851-2250-4cfd-969d-f8c9ad9f9ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:56.615102 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d6851-2250-4cfd-969d-f8c9ad9f9ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:56.615377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d6851-2250-4cfd-969d-f8c9ad9f9ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:56.617186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d6851-2250-4cfd-969d-f8c9ad9f9ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:56.620311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d6851-2250-4cfd-969d-f8c9ad9f9ff2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:56.627071 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50140e862037436581f713b8f1a38edb'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:56.642955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1031/2038] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:56 2022] DELETE /identity/v3/users/50140e862037436581f713b8f1a38edb => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:56.680759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1008/2039] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:56 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:56.684288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:56.685859 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:56.798960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51db49-0a03-46af-accb-9cd0a5f8e24d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:56.915889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295e4f2-4c80-411b-aa85-91bb90f822c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:56.917264 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295e4f2-4c80-411b-aa85-91bb90f822c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:56.917631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295e4f2-4c80-411b-aa85-91bb90f822c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:56.917919 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295e4f2-4c80-411b-aa85-91bb90f822c9 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:56.918649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295e4f2-4c80-411b-aa85-91bb90f822c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:56.922361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295e4f2-4c80-411b-aa85-91bb90f822c9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:56.964408 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1032/2040] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:56 2022] POST /identity/v3/users => generated 300 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:56.966807 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d6851-2250-4cfd-969d-f8c9ad9f9ff2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:56.980932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d6851-2250-4cfd-969d-f8c9ad9f9ff2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:56.987658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036b603-0bbb-4415-b653-fa3da8b4b0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:56.990841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036b603-0bbb-4415-b653-fa3da8b4b0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:56.991282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036b603-0bbb-4415-b653-fa3da8b4b0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:56.991661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036b603-0bbb-4415-b653-fa3da8b4b0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:56.992754 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036b603-0bbb-4415-b653-fa3da8b4b0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:56.994682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036b603-0bbb-4415-b653-fa3da8b4b0b3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.011986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1009/2041] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:57.014305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295e4f2-4c80-411b-aa85-91bb90f822c9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.038116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295e4f2-4c80-411b-aa85-91bb90f822c9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.062633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ae96a-cde9-422b-987c-27db8f0ee732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.064352 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ae96a-cde9-422b-987c-27db8f0ee732 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.066312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ae96a-cde9-422b-987c-27db8f0ee732 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.066743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ae96a-cde9-422b-987c-27db8f0ee732 admin admin] PATH_INFO: `/v3/projects/385da98486d641e7bf9ee8042a527643/users/10ed59d75ea94ee98e09a0b06485e7a7/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.099456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ae96a-cde9-422b-987c-27db8f0ee732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=385da98486d641e7bf9ee8042a527643, user_id=10ed59d75ea94ee98e09a0b06485e7a7, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:57.102958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ae96a-cde9-422b-987c-27db8f0ee732 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.118277 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f578f-603f-46f9-bed1-77be10132900 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.118936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f578f-603f-46f9-bed1-77be10132900 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.118936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f578f-603f-46f9-bed1-77be10132900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.136344 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1033/2042] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:57 2022] PUT /identity/v3/projects/385da98486d641e7bf9ee8042a527643/users/10ed59d75ea94ee98e09a0b06485e7a7/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:57.137251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ae96a-cde9-422b-987c-27db8f0ee732 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.140381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-793ae96a-cde9-422b-987c-27db8f0ee732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:57.166858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1b9f578f-603f-46f9-bed1-77be10132900 None None] MFA Rules not processed for user `7c767bec3f4c4c9c8698afce8e40fd19`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:57.168168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1b9f578f-603f-46f9-bed1-77be10132900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:57.222324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:57.231524 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ae96a-cde9-422b-987c-27db8f0ee732 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.239768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.240685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.240974 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.241274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] PATH_INFO: `/v3/projects/ded8ead8a1f74e1e806ab0c3c1bc953c` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.243543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ded8ead8a1f74e1e806ab0c3c1bc953c)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:57.245866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.262765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1010/2043] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:57.267619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f578f-603f-46f9-bed1-77be10132900 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.286497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f578f-603f-46f9-bed1-77be10132900 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.304265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ded8ead8a1f74e1e806ab0c3c1bc953c', 'request_id': 'req-287b892c-d959-4d28-88e4-734915613224'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:57.317969 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ded8ead8a1f74e1e806ab0c3c1bc953c', 'request_id': 'req-287b892c-d959-4d28-88e4-734915613224'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:57.327996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ded8ead8a1f74e1e806ab0c3c1bc953c', 'request_id': 'req-287b892c-d959-4d28-88e4-734915613224'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:57.344809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:57.345036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] The token cache is being invalidate because project ded8ead8a1f74e1e806ab0c3c1bc953c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:57.345283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:57.346321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:57.349283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1034/2044] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:57 2022] DELETE /identity/v3/projects/ded8ead8a1f74e1e806ab0c3c1bc953c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:57.350229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.352633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:57.354174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c748a-0ac1-4222-97d0-9a2d5db26fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.355170 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c748a-0ac1-4222-97d0-9a2d5db26fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.356076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c748a-0ac1-4222-97d0-9a2d5db26fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.356732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c748a-0ac1-4222-97d0-9a2d5db26fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.357575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c748a-0ac1-4222-97d0-9a2d5db26fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:57.359942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c748a-0ac1-4222-97d0-9a2d5db26fc7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.368380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b892c-d959-4d28-88e4-734915613224 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.375769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1011/2045] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:57.375769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dccc1-5c32-46d2-8f87-79d2feb7330f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.375769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dccc1-5c32-46d2-8f87-79d2feb7330f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.375769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dccc1-5c32-46d2-8f87-79d2feb7330f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.379125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0e1e1-8bce-44d5-b398-b76ef48592a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.379910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0e1e1-8bce-44d5-b398-b76ef48592a8 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.380239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0e1e1-8bce-44d5-b398-b76ef48592a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.380520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0e1e1-8bce-44d5-b398-b76ef48592a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.381321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0e1e1-8bce-44d5-b398-b76ef48592a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:57.385163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0e1e1-8bce-44d5-b398-b76ef48592a8 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.397426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-140dccc1-5c32-46d2-8f87-79d2feb7330f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:57.398663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-140dccc1-5c32-46d2-8f87-79d2feb7330f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:57.412088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1012/2046] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:57.413246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dccc1-5c32-46d2-8f87-79d2feb7330f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.414811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-140dccc1-5c32-46d2-8f87-79d2feb7330f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:57.434507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dccc1-5c32-46d2-8f87-79d2feb7330f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.437092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1035/2047] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:57 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:57.438741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb24d5-a95c-463b-9fc3-f04b6d2c9250 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.439233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb24d5-a95c-463b-9fc3-f04b6d2c9250 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.440041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb24d5-a95c-463b-9fc3-f04b6d2c9250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.468853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-fdeb24d5-a95c-463b-9fc3-f04b6d2c9250 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:57.470246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-fdeb24d5-a95c-463b-9fc3-f04b6d2c9250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:57.533680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f160a-4546-4c49-acce-9cfd0f0617a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.534815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f160a-4546-4c49-acce-9cfd0f0617a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.535350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f160a-4546-4c49-acce-9cfd0f0617a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.535679 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f160a-4546-4c49-acce-9cfd0f0617a6 admin admin] PATH_INFO: `/v3/projects/385da98486d641e7bf9ee8042a527643/users/10ed59d75ea94ee98e09a0b06485e7a7/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.537970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f160a-4546-4c49-acce-9cfd0f0617a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=385da98486d641e7bf9ee8042a527643, user_id=10ed59d75ea94ee98e09a0b06485e7a7, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:57.540820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f160a-4546-4c49-acce-9cfd0f0617a6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.555568 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1036/2049] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:57.556070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1013/2049] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:57 2022] PUT /identity/v3/projects/385da98486d641e7bf9ee8042a527643/users/10ed59d75ea94ee98e09a0b06485e7a7/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:57.556666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb24d5-a95c-463b-9fc3-f04b6d2c9250 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.559924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f160a-4546-4c49-acce-9cfd0f0617a6 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.572052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb24d5-a95c-463b-9fc3-f04b6d2c9250 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.651341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c51e34-1e5c-4a41-90db-b9a0622dc41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.651711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f160a-4546-4c49-acce-9cfd0f0617a6 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.652198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c51e34-1e5c-4a41-90db-b9a0622dc41a admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.652486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c51e34-1e5c-4a41-90db-b9a0622dc41a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.652773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c51e34-1e5c-4a41-90db-b9a0622dc41a admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.653567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c51e34-1e5c-4a41-90db-b9a0622dc41a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:57.655491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c51e34-1e5c-4a41-90db-b9a0622dc41a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.656450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f9b57-2b77-42ee-8e5f-8c0fb2ea2c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.657119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f9b57-2b77-42ee-8e5f-8c0fb2ea2c2f admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.657399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f9b57-2b77-42ee-8e5f-8c0fb2ea2c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.657672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f9b57-2b77-42ee-8e5f-8c0fb2ea2c2f admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.658556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f9b57-2b77-42ee-8e5f-8c0fb2ea2c2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:49:57.658865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f9b57-2b77-42ee-8e5f-8c0fb2ea2c2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:57.660522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f9b57-2b77-42ee-8e5f-8c0fb2ea2c2f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.664331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:57.678715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1014/2050] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:49:57.678715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf63ea5-c594-41f5-87e3-ab5ff405e54e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.678715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf63ea5-c594-41f5-87e3-ab5ff405e54e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.678715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf63ea5-c594-41f5-87e3-ab5ff405e54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.682547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1037/2051] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:57 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:57.683398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c51e34-1e5c-4a41-90db-b9a0622dc41a admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.692883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c51e34-1e5c-4a41-90db-b9a0622dc41a admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.698491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f31c65-e4f1-44d4-8015-a95102d2f608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.699965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f31c65-e4f1-44d4-8015-a95102d2f608 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.699965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f31c65-e4f1-44d4-8015-a95102d2f608 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.699965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f31c65-e4f1-44d4-8015-a95102d2f608 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.700456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f31c65-e4f1-44d4-8015-a95102d2f608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:57.702577 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f31c65-e4f1-44d4-8015-a95102d2f608 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.710175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9cf63ea5-c594-41f5-87e3-ab5ff405e54e None None] MFA Rules not processed for user `10ed59d75ea94ee98e09a0b06485e7a7`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:57.711551 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9cf63ea5-c594-41f5-87e3-ab5ff405e54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:57.738074 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1038/2052] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:57 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:57.739266 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f31c65-e4f1-44d4-8015-a95102d2f608 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.778800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:57.789749 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1015/2053] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:57.790475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf63ea5-c594-41f5-87e3-ab5ff405e54e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.800978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf63ea5-c594-41f5-87e3-ab5ff405e54e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.802911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f31c65-e4f1-44d4-8015-a95102d2f608 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.805884 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c44799-e6a0-430a-9d0a-18e0e34f57bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.806498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c44799-e6a0-430a-9d0a-18e0e34f57bb admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.806784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c44799-e6a0-430a-9d0a-18e0e34f57bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.807094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c44799-e6a0-430a-9d0a-18e0e34f57bb admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.807579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b0c26-2ed2-4f3a-b060-1832c993884d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.807796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c44799-e6a0-430a-9d0a-18e0e34f57bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:57.808280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b0c26-2ed2-4f3a-b060-1832c993884d admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.808563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b0c26-2ed2-4f3a-b060-1832c993884d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.808842 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b0c26-2ed2-4f3a-b060-1832c993884d admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.809849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c44799-e6a0-430a-9d0a-18e0e34f57bb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.809951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b0c26-2ed2-4f3a-b060-1832c993884d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:57.811453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b0c26-2ed2-4f3a-b060-1832c993884d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.821470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1039/2054] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:57.824091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b0c26-2ed2-4f3a-b060-1832c993884d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.825281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1b9b0c26-2ed2-4f3a-b060-1832c993884d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:57.837556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1016/2055] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:57 2022] POST /identity/v3/users => generated 295 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:57.837556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c44799-e6a0-430a-9d0a-18e0e34f57bb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.847621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c44799-e6a0-430a-9d0a-18e0e34f57bb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.852566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdda2c-1c19-4d1a-8a0e-e772fd7e8c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.853470 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdda2c-1c19-4d1a-8a0e-e772fd7e8c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.853765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdda2c-1c19-4d1a-8a0e-e772fd7e8c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.854063 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdda2c-1c19-4d1a-8a0e-e772fd7e8c20 admin admin] PATH_INFO: `/v3/projects/aee2b9c236af4831a343a10c84445ee7/users/9d9d4b27802846fe9669b938b6a9e215/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.869871 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debdda2c-1c19-4d1a-8a0e-e772fd7e8c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee2b9c236af4831a343a10c84445ee7, user_id=9d9d4b27802846fe9669b938b6a9e215, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:57.872628 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debdda2c-1c19-4d1a-8a0e-e772fd7e8c20 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.886859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1017/2056] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:57 2022] PUT /identity/v3/projects/aee2b9c236af4831a343a10c84445ee7/users/9d9d4b27802846fe9669b938b6a9e215/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:57.888257 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdda2c-1c19-4d1a-8a0e-e772fd7e8c20 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:57.892717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b0c26-2ed2-4f3a-b060-1832c993884d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.952621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa037f7-e49f-4762-b53a-fd31a2579c0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.953283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa037f7-e49f-4762-b53a-fd31a2579c0d service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.953568 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa037f7-e49f-4762-b53a-fd31a2579c0d service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.953975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa037f7-e49f-4762-b53a-fd31a2579c0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.968027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdda2c-1c19-4d1a-8a0e-e772fd7e8c20 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:57.974148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7dcaa-edaf-4c04-b0bc-f1695802b278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:57.975108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7dcaa-edaf-4c04-b0bc-f1695802b278 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:57.975399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7dcaa-edaf-4c04-b0bc-f1695802b278 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:57.975676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7dcaa-edaf-4c04-b0bc-f1695802b278 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:57.976499 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7dcaa-edaf-4c04-b0bc-f1695802b278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:57.978890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7dcaa-edaf-4c04-b0bc-f1695802b278 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:57.991816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1018/2057] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:57.992859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7dcaa-edaf-4c04-b0bc-f1695802b278 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:58.003081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:58.016077 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7dcaa-edaf-4c04-b0bc-f1695802b278 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:58.021313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f777c96-e012-4e52-8658-dbd2263c8fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:58.021993 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f777c96-e012-4e52-8658-dbd2263c8fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.022274 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f777c96-e012-4e52-8658-dbd2263c8fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.022554 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f777c96-e012-4e52-8658-dbd2263c8fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.023281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f777c96-e012-4e52-8658-dbd2263c8fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:58.025066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f777c96-e012-4e52-8658-dbd2263c8fa8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:58.028020 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa037f7-e49f-4762-b53a-fd31a2579c0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:58.029511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa037f7-e49f-4762-b53a-fd31a2579c0d service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:58.035392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1019/2058] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:58.036260 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f777c96-e012-4e52-8658-dbd2263c8fa8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:58.039699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1040/2059] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:58.040425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa037f7-e49f-4762-b53a-fd31a2579c0d service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:58.046232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f777c96-e012-4e52-8658-dbd2263c8fa8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:58.050251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa037f7-e49f-4762-b53a-fd31a2579c0d service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:58.051042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d00432-4282-40c8-b502-88d93bf7cdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:58.051712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d00432-4282-40c8-b502-88d93bf7cdda admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.052092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d00432-4282-40c8-b502-88d93bf7cdda admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.052309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d00432-4282-40c8-b502-88d93bf7cdda admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.053030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d00432-4282-40c8-b502-88d93bf7cdda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:58.054629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d00432-4282-40c8-b502-88d93bf7cdda admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:58.055523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa368c-e21b-4e8b-bba1-c8c523508cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:58.056546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa368c-e21b-4e8b-bba1-c8c523508cf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.056807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa368c-e21b-4e8b-bba1-c8c523508cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.057090 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa368c-e21b-4e8b-bba1-c8c523508cf2 admin admin] PATH_INFO: `/v3/projects/aee2b9c236af4831a343a10c84445ee7/users/9d9d4b27802846fe9669b938b6a9e215/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.059538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa368c-e21b-4e8b-bba1-c8c523508cf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee2b9c236af4831a343a10c84445ee7, user_id=9d9d4b27802846fe9669b938b6a9e215, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:58.062768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa368c-e21b-4e8b-bba1-c8c523508cf2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:58.071572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1020/2060] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:58.072093 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d00432-4282-40c8-b502-88d93bf7cdda admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:58.080257 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1041/2061] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:58 2022] PUT /identity/v3/projects/aee2b9c236af4831a343a10c84445ee7/users/9d9d4b27802846fe9669b938b6a9e215/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:58.084124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d00432-4282-40c8-b502-88d93bf7cdda admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:58.176980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102b457-2e1d-4f0f-934f-9d5d09eaaee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:58.179057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102b457-2e1d-4f0f-934f-9d5d09eaaee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.179057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102b457-2e1d-4f0f-934f-9d5d09eaaee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.179305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102b457-2e1d-4f0f-934f-9d5d09eaaee2 admin admin] PATH_INFO: `/v3/projects/ed9c5b21c8e54600a913ad9be5883678/users/ecbfc005fb654635bda4b341ec71ce98/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.200641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102b457-2e1d-4f0f-934f-9d5d09eaaee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed9c5b21c8e54600a913ad9be5883678, user_id=ecbfc005fb654635bda4b341ec71ce98, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:58.204475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102b457-2e1d-4f0f-934f-9d5d09eaaee2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:58.227738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1021/2062] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:58 2022] PUT /identity/v3/projects/ed9c5b21c8e54600a913ad9be5883678/users/ecbfc005fb654635bda4b341ec71ce98/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:58.239871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5b45e-857e-4fcc-9387-b6a99d151c12 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.240238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5b45e-857e-4fcc-9387-b6a99d151c12 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.240561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5b45e-857e-4fcc-9387-b6a99d151c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.281881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-65f5b45e-857e-4fcc-9387-b6a99d151c12 None None] MFA Rules not processed for user `ecbfc005fb654635bda4b341ec71ce98`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:58.284029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-65f5b45e-857e-4fcc-9387-b6a99d151c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:58.364048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1042/2063] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:58.367051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102b457-2e1d-4f0f-934f-9d5d09eaaee2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:58.384310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:58.444819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102b457-2e1d-4f0f-934f-9d5d09eaaee2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:58.451685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d37dc-d1b5-48ad-b008-52a4c7237cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:58.452448 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d37dc-d1b5-48ad-b008-52a4c7237cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.452802 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d37dc-d1b5-48ad-b008-52a4c7237cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.453084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d37dc-d1b5-48ad-b008-52a4c7237cf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.460999 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d37dc-d1b5-48ad-b008-52a4c7237cf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:58.469036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d37dc-d1b5-48ad-b008-52a4c7237cf8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:58.520886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1022/2064] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:49:58 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:58.522605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5b45e-857e-4fcc-9387-b6a99d151c12 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:58.540126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5b45e-857e-4fcc-9387-b6a99d151c12 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:58.606835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1965d9-9cd0-4757-8e2c-e698ea808619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:58.609215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1965d9-9cd0-4757-8e2c-e698ea808619 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.609215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1965d9-9cd0-4757-8e2c-e698ea808619 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.609215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1965d9-9cd0-4757-8e2c-e698ea808619 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.609501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1965d9-9cd0-4757-8e2c-e698ea808619 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:58.611692 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1965d9-9cd0-4757-8e2c-e698ea808619 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:58.637489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1043/2065] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:49:58 2022] POST /identity/v3/users => generated 300 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:49:58.638373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d37dc-d1b5-48ad-b008-52a4c7237cf8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:58.650709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d37dc-d1b5-48ad-b008-52a4c7237cf8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:58.657026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5730c77-ea71-493b-ae15-47734959bf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:58.657879 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5730c77-ea71-493b-ae15-47734959bf6b admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.658235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5730c77-ea71-493b-ae15-47734959bf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.658728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5730c77-ea71-493b-ae15-47734959bf6b admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.659531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5730c77-ea71-493b-ae15-47734959bf6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:58.661400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5730c77-ea71-493b-ae15-47734959bf6b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:58.680462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1023/2066] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:49:58.684644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1965d9-9cd0-4757-8e2c-e698ea808619 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:58.697226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1965d9-9cd0-4757-8e2c-e698ea808619 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:58.702929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c769cfc-2f10-4415-a576-b5e0ddb70bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:58.703960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c769cfc-2f10-4415-a576-b5e0ddb70bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.704345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c769cfc-2f10-4415-a576-b5e0ddb70bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.704688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c769cfc-2f10-4415-a576-b5e0ddb70bcf admin admin] PATH_INFO: `/v3/projects/ac8073f8db724e8db91e9bd1ffd3c160/users/e7781accbe0e44edb7e51575ca793087/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.726932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c769cfc-2f10-4415-a576-b5e0ddb70bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac8073f8db724e8db91e9bd1ffd3c160, user_id=e7781accbe0e44edb7e51575ca793087, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:58.732759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c769cfc-2f10-4415-a576-b5e0ddb70bcf admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:58.755856 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5730c77-ea71-493b-ae15-47734959bf6b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:58.758996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1044/2067] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:58 2022] PUT /identity/v3/projects/ac8073f8db724e8db91e9bd1ffd3c160/users/e7781accbe0e44edb7e51575ca793087/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:58.762944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c769cfc-2f10-4415-a576-b5e0ddb70bcf admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:58.777038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5730c77-ea71-493b-ae15-47734959bf6b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:58.793117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:58.848927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c769cfc-2f10-4415-a576-b5e0ddb70bcf admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:58.854046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03a953-baf3-435f-b2db-4691d0b708c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:58.854740 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03a953-baf3-435f-b2db-4691d0b708c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.855029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03a953-baf3-435f-b2db-4691d0b708c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.855473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03a953-baf3-435f-b2db-4691d0b708c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.856264 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03a953-baf3-435f-b2db-4691d0b708c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:58.857968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03a953-baf3-435f-b2db-4691d0b708c7 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:58.873432 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1045/2068] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:49:58.877349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03a953-baf3-435f-b2db-4691d0b708c7 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:58.878515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f5e29-6e64-4a85-ba12-8809001bf66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:58.880900 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f5e29-6e64-4a85-ba12-8809001bf66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.882710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f5e29-6e64-4a85-ba12-8809001bf66d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.883233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f5e29-6e64-4a85-ba12-8809001bf66d admin admin] PATH_INFO: `/v3/users/8fa2d0c8c4124774be30fa97e36c94dd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.885839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f5e29-6e64-4a85-ba12-8809001bf66d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fa2d0c8c4124774be30fa97e36c94dd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:58.888141 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f5e29-6e64-4a85-ba12-8809001bf66d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:58.902907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03a953-baf3-435f-b2db-4691d0b708c7 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:58.908084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79480024-7c89-4543-8e74-116eefb57204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:58.908996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480024-7c89-4543-8e74-116eefb57204 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.909280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480024-7c89-4543-8e74-116eefb57204 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.909664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480024-7c89-4543-8e74-116eefb57204 admin admin] PATH_INFO: `/v3/projects/ac8073f8db724e8db91e9bd1ffd3c160/users/e7781accbe0e44edb7e51575ca793087/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.911678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79480024-7c89-4543-8e74-116eefb57204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac8073f8db724e8db91e9bd1ffd3c160, user_id=e7781accbe0e44edb7e51575ca793087, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:58.924468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79480024-7c89-4543-8e74-116eefb57204 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:58.948079 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1046/2069] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:49:58 2022] PUT /identity/v3/projects/ac8073f8db724e8db91e9bd1ffd3c160/users/e7781accbe0e44edb7e51575ca793087/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:58.950300 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b2e3c-9062-407b-80c8-b533be00651c None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:58.950595 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b2e3c-9062-407b-80c8-b533be00651c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:58.950906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b2e3c-9062-407b-80c8-b533be00651c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:58.974318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-869f5e29-6e64-4a85-ba12-8809001bf66d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fa2d0c8c4124774be30fa97e36c94dd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:58.983430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-869f5e29-6e64-4a85-ba12-8809001bf66d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fa2d0c8c4124774be30fa97e36c94dd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:58.983622 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-919b2e3c-9062-407b-80c8-b533be00651c None None] MFA Rules not processed for user `e7781accbe0e44edb7e51575ca793087`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:58.984262 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-869f5e29-6e64-4a85-ba12-8809001bf66d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fa2d0c8c4124774be30fa97e36c94dd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:58.984870 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-919b2e3c-9062-407b-80c8-b533be00651c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:58.995682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-869f5e29-6e64-4a85-ba12-8809001bf66d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fa2d0c8c4124774be30fa97e36c94dd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:59.019716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-869f5e29-6e64-4a85-ba12-8809001bf66d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fa2d0c8c4124774be30fa97e36c94dd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:59.030327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1024/2070] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:58 2022] DELETE /identity/v3/users/8fa2d0c8c4124774be30fa97e36c94dd => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:59.031801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029f877-6787-4d84-a446-5d46437a7f00 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:59.032317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029f877-6787-4d84-a446-5d46437a7f00 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:59.032601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029f877-6787-4d84-a446-5d46437a7f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:59.050666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:59.078960 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4029f877-6787-4d84-a446-5d46437a7f00 None None] MFA Rules not processed for user `9d9d4b27802846fe9669b938b6a9e215`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:59.081393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4029f877-6787-4d84-a446-5d46437a7f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:59.092074 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1047/2071] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:59.097954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedd956-ba1d-430b-a7e3-bf4ca1d2ecc1 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:59.097954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedd956-ba1d-430b-a7e3-bf4ca1d2ecc1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:59.098235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedd956-ba1d-430b-a7e3-bf4ca1d2ecc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:59.130305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-faedd956-ba1d-430b-a7e3-bf4ca1d2ecc1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:59.131823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-faedd956-ba1d-430b-a7e3-bf4ca1d2ecc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:59.190680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1025/2072] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:59.191971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029f877-6787-4d84-a446-5d46437a7f00 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:59.193068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4029f877-6787-4d84-a446-5d46437a7f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:59.205672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1048/2073] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:59.206677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faedd956-ba1d-430b-a7e3-bf4ca1d2ecc1 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:59.209706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-faedd956-ba1d-430b-a7e3-bf4ca1d2ecc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:59.237409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:59.258391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029f877-6787-4d84-a446-5d46437a7f00 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:59.262076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17d00e-3fd0-4b00-bc94-7c67c8cc2dda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:59.262799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17d00e-3fd0-4b00-bc94-7c67c8cc2dda service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:59.263147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17d00e-3fd0-4b00-bc94-7c67c8cc2dda service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:59.263492 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17d00e-3fd0-4b00-bc94-7c67c8cc2dda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:59.268221 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faedd956-ba1d-430b-a7e3-bf4ca1d2ecc1 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:59.273982 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004460d-a04e-421f-ab31-0b91f4619edb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:59.274639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004460d-a04e-421f-ab31-0b91f4619edb service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:59.275069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004460d-a04e-421f-ab31-0b91f4619edb service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:59.275350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004460d-a04e-421f-ab31-0b91f4619edb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:59.277445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7004460d-a04e-421f-ab31-0b91f4619edb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:59.329938 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17d00e-3fd0-4b00-bc94-7c67c8cc2dda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:59.331617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d17d00e-3fd0-4b00-bc94-7c67c8cc2dda service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:59.341300 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004460d-a04e-421f-ab31-0b91f4619edb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:59.341437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1026/2074] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:59.342300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17d00e-3fd0-4b00-bc94-7c67c8cc2dda service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:59.342800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004460d-a04e-421f-ab31-0b91f4619edb service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:59.343846 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3d17d00e-3fd0-4b00-bc94-7c67c8cc2dda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:59.352838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1049/2075] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:59.396482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:59.417459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d17d00e-3fd0-4b00-bc94-7c67c8cc2dda service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:59.421506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de358e-574b-4405-ad81-98bd4554447e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:59.422198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de358e-574b-4405-ad81-98bd4554447e service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:59.422515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de358e-574b-4405-ad81-98bd4554447e service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:59.422791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de358e-574b-4405-ad81-98bd4554447e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:59.432847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de358e-574b-4405-ad81-98bd4554447e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:59.434309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de358e-574b-4405-ad81-98bd4554447e service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:59.448381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1027/2076] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:49:59.514818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004460d-a04e-421f-ab31-0b91f4619edb service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:59.516293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7004460d-a04e-421f-ab31-0b91f4619edb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:59.535441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004460d-a04e-421f-ab31-0b91f4619edb service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:59.545321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:59.547027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:59.548332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:59.548712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] PATH_INFO: `/v3/users/ecbfc005fb654635bda4b341ec71ce98` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:59.551102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecbfc005fb654635bda4b341ec71ce98)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:59.553880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:59.617377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecbfc005fb654635bda4b341ec71ce98'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:59.618584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecbfc005fb654635bda4b341ec71ce98'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:59.627625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecbfc005fb654635bda4b341ec71ce98'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:59.636028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecbfc005fb654635bda4b341ec71ce98'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:59.652125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecbfc005fb654635bda4b341ec71ce98'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:59.662531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1050/2077] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:59 2022] DELETE /identity/v3/users/ecbfc005fb654635bda4b341ec71ce98 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:59.664302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdbda0-e432-484c-b630-1a9824f9c066 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:59.664671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdbda0-e432-484c-b630-1a9824f9c066 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:59.665018 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdbda0-e432-484c-b630-1a9824f9c066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:59.699809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-16fdbda0-e432-484c-b630-1a9824f9c066 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:49:59.701750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-16fdbda0-e432-484c-b630-1a9824f9c066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:59.785563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:59.787064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:59.788387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1028/2078] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:49:59.798795 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdbda0-e432-484c-b630-1a9824f9c066 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:59.800416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-16fdbda0-e432-484c-b630-1a9824f9c066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:59.801977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f515971-4281-4f56-b167-a4888a31c7cd admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:59.806948 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:59.807825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:59.808163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:59.808441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] PATH_INFO: `/v3/projects/e6759753399645b99c721a54ca969214` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:59.809796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6759753399645b99c721a54ca969214)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:49:59.811718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:49:59.858768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:49:59.862231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6759753399645b99c721a54ca969214', 'request_id': 'req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:59.863203 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdbda0-e432-484c-b630-1a9824f9c066 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:49:59.873563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6759753399645b99c721a54ca969214', 'request_id': 'req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:59.880612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6759753399645b99c721a54ca969214', 'request_id': 'req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:59.891730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] The token cache is being invalidate because project e6759753399645b99c721a54ca969214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:49:59.891945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:59.892660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:49:59.894346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1051/2079] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:49:59 2022] DELETE /identity/v3/projects/e6759753399645b99c721a54ca969214 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:49:59.905019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:49:59.906415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:49:59.920340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91dda0-03dc-4afb-b056-53db3e914e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:49:59.920983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91dda0-03dc-4afb-b056-53db3e914e87 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:49:59.921263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91dda0-03dc-4afb-b056-53db3e914e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:49:59.921535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91dda0-03dc-4afb-b056-53db3e914e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:49:59.925513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fc91dda0-03dc-4afb-b056-53db3e914e87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:00.006816 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb709d2e-5ebd-4e6b-a2c9-1e37fcc64b05 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:00.011511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:00.015095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:00.016034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:00.016384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:00.016797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] PATH_INFO: `/v3/users/10ed59d75ea94ee98e09a0b06485e7a7` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:00.018054 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10ed59d75ea94ee98e09a0b06485e7a7)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:00.020513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:00.034203 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91dda0-03dc-4afb-b056-53db3e914e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:00.035733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91dda0-03dc-4afb-b056-53db3e914e87 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:00.052198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1029/2080] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:00.081239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10ed59d75ea94ee98e09a0b06485e7a7'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.082000 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10ed59d75ea94ee98e09a0b06485e7a7'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.095243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10ed59d75ea94ee98e09a0b06485e7a7'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.105555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10ed59d75ea94ee98e09a0b06485e7a7'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.129274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10ed59d75ea94ee98e09a0b06485e7a7'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.148924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1052/2081] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:49:59 2022] DELETE /identity/v3/users/10ed59d75ea94ee98e09a0b06485e7a7 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:00.379166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91dda0-03dc-4afb-b056-53db3e914e87 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:00.381517 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fc91dda0-03dc-4afb-b056-53db3e914e87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:00.397001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91dda0-03dc-4afb-b056-53db3e914e87 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:00.486770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:00.487645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:00.487923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:00.488225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] PATH_INFO: `/v3/projects/ed9c5b21c8e54600a913ad9be5883678` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:00.489401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed9c5b21c8e54600a913ad9be5883678)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:00.491323 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:00.537915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed9c5b21c8e54600a913ad9be5883678', 'request_id': 'req-60fdfe66-5593-42a4-9418-f8c4cde944b0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.547092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed9c5b21c8e54600a913ad9be5883678', 'request_id': 'req-60fdfe66-5593-42a4-9418-f8c4cde944b0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.561840 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed9c5b21c8e54600a913ad9be5883678', 'request_id': 'req-60fdfe66-5593-42a4-9418-f8c4cde944b0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.580780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] The token cache is being invalidate because project ed9c5b21c8e54600a913ad9be5883678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:00.581514 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.583755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.585155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:00.586628 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:00.596247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1030/2082] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:00 2022] DELETE /identity/v3/projects/ed9c5b21c8e54600a913ad9be5883678 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:00.604712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:00.606411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:00.612189 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b42f3-1961-4a5f-8215-54169bf9c6f0 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:00.679853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:00.689003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28445290-8edd-456a-84d1-68d4c112807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:00.690085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445290-8edd-456a-84d1-68d4c112807e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:00.690549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445290-8edd-456a-84d1-68d4c112807e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:00.690992 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445290-8edd-456a-84d1-68d4c112807e admin admin] PATH_INFO: `/v3/projects/385da98486d641e7bf9ee8042a527643` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:00.692434 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28445290-8edd-456a-84d1-68d4c112807e admin admin] RBAC: Authorizing `identity:delete_project(project_id=385da98486d641e7bf9ee8042a527643)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:00.694403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdfe66-5593-42a4-9418-f8c4cde944b0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:00.694584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28445290-8edd-456a-84d1-68d4c112807e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:00.699526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed427599-6162-4c30-9563-898417ff9c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:00.700476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed427599-6162-4c30-9563-898417ff9c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:00.700753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed427599-6162-4c30-9563-898417ff9c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:00.701054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed427599-6162-4c30-9563-898417ff9c39 admin admin] PATH_INFO: `/v3/users/e7781accbe0e44edb7e51575ca793087` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:00.702546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed427599-6162-4c30-9563-898417ff9c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7781accbe0e44edb7e51575ca793087)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:00.704547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed427599-6162-4c30-9563-898417ff9c39 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:00.745575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:00.751906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-28445290-8edd-456a-84d1-68d4c112807e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '385da98486d641e7bf9ee8042a527643', 'request_id': 'req-28445290-8edd-456a-84d1-68d4c112807e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.761579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-28445290-8edd-456a-84d1-68d4c112807e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '385da98486d641e7bf9ee8042a527643', 'request_id': 'req-28445290-8edd-456a-84d1-68d4c112807e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.771134 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-28445290-8edd-456a-84d1-68d4c112807e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '385da98486d641e7bf9ee8042a527643', 'request_id': 'req-28445290-8edd-456a-84d1-68d4c112807e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.782028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ed427599-6162-4c30-9563-898417ff9c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7781accbe0e44edb7e51575ca793087'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.782768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-28445290-8edd-456a-84d1-68d4c112807e admin admin] The token cache is being invalidate because project 385da98486d641e7bf9ee8042a527643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:00.783084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-28445290-8edd-456a-84d1-68d4c112807e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.783736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-28445290-8edd-456a-84d1-68d4c112807e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.785534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1053/2083] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:00 2022] DELETE /identity/v3/projects/385da98486d641e7bf9ee8042a527643 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:00.790846 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b287e-d6e7-4b50-b0be-fa0a8d945e03 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:00.791272 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b287e-d6e7-4b50-b0be-fa0a8d945e03 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:00.791661 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b287e-d6e7-4b50-b0be-fa0a8d945e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:00.792920 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ed427599-6162-4c30-9563-898417ff9c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7781accbe0e44edb7e51575ca793087'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.793545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ed427599-6162-4c30-9563-898417ff9c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7781accbe0e44edb7e51575ca793087'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.801776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ed427599-6162-4c30-9563-898417ff9c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7781accbe0e44edb7e51575ca793087'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.809730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d74b287e-d6e7-4b50-b0be-fa0a8d945e03 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:00.810944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d74b287e-d6e7-4b50-b0be-fa0a8d945e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:00.814065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ed427599-6162-4c30-9563-898417ff9c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7781accbe0e44edb7e51575ca793087'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:00.823126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1031/2084] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:00 2022] DELETE /identity/v3/users/e7781accbe0e44edb7e51575ca793087 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:00.869876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1054/2085] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:50:00.877533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506e811-aaa1-414f-bde9-c096a4a5ae59 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:00.877815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506e811-aaa1-414f-bde9-c096a4a5ae59 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:00.878121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506e811-aaa1-414f-bde9-c096a4a5ae59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:00.897128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1506e811-aaa1-414f-bde9-c096a4a5ae59 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:00.898327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1506e811-aaa1-414f-bde9-c096a4a5ae59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:00.962886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1032/2086] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:50:00.967805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b287e-d6e7-4b50-b0be-fa0a8d945e03 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:00.978881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b287e-d6e7-4b50-b0be-fa0a8d945e03 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:00.984708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa27d3-53a0-474c-8e5f-9a6d60812879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:00.985383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa27d3-53a0-474c-8e5f-9a6d60812879 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:00.985674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa27d3-53a0-474c-8e5f-9a6d60812879 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:00.985953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa27d3-53a0-474c-8e5f-9a6d60812879 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:00.986690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa27d3-53a0-474c-8e5f-9a6d60812879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:50:00.987005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa27d3-53a0-474c-8e5f-9a6d60812879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:00.988730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa27d3-53a0-474c-8e5f-9a6d60812879 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.002394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1055/2087] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:50:01.006742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506e811-aaa1-414f-bde9-c096a4a5ae59 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.015997 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506e811-aaa1-414f-bde9-c096a4a5ae59 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.021762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a7857-5dd5-4f82-a869-cb4f2675e6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.022442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a7857-5dd5-4f82-a869-cb4f2675e6ca admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.022719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a7857-5dd5-4f82-a869-cb4f2675e6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.023009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a7857-5dd5-4f82-a869-cb4f2675e6ca admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.023671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a7857-5dd5-4f82-a869-cb4f2675e6ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.025628 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a7857-5dd5-4f82-a869-cb4f2675e6ca admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.056059 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1033/2088] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:01 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:01.060462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa27d3-53a0-474c-8e5f-9a6d60812879 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.070776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa27d3-53a0-474c-8e5f-9a6d60812879 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.099666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a7857-5dd5-4f82-a869-cb4f2675e6ca admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.101420 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e46a7857-5dd5-4f82-a869-cb4f2675e6ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:01.140569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb4ae5-295b-4abe-913d-1fa3a44628a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.141247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb4ae5-295b-4abe-913d-1fa3a44628a1 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.141680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb4ae5-295b-4abe-913d-1fa3a44628a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.141959 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb4ae5-295b-4abe-913d-1fa3a44628a1 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.142814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb4ae5-295b-4abe-913d-1fa3a44628a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.145945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb4ae5-295b-4abe-913d-1fa3a44628a1 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.174199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:01.181759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a7857-5dd5-4f82-a869-cb4f2675e6ca admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.187261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1056/2089] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:01 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:01.189318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb4ae5-295b-4abe-913d-1fa3a44628a1 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.200694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb4ae5-295b-4abe-913d-1fa3a44628a1 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.202685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e67a12-8c81-4c70-8dae-0a468535469d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.205236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e67a12-8c81-4c70-8dae-0a468535469d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.205667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e67a12-8c81-4c70-8dae-0a468535469d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.206028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e67a12-8c81-4c70-8dae-0a468535469d admin admin] PATH_INFO: `/v3/projects/ac8073f8db724e8db91e9bd1ffd3c160` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.206807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210f862-7841-401b-9bd8-d6429a48e097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.207456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210f862-7841-401b-9bd8-d6429a48e097 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.207798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210f862-7841-401b-9bd8-d6429a48e097 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.207939 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e67a12-8c81-4c70-8dae-0a468535469d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac8073f8db724e8db91e9bd1ffd3c160)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.208270 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210f862-7841-401b-9bd8-d6429a48e097 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.209388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210f862-7841-401b-9bd8-d6429a48e097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.210024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e67a12-8c81-4c70-8dae-0a468535469d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.211042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210f862-7841-401b-9bd8-d6429a48e097 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.222986 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1057/2090] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:01.227514 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210f862-7841-401b-9bd8-d6429a48e097 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.239091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210f862-7841-401b-9bd8-d6429a48e097 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.245612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc944be6-c533-4147-9f2d-99e3f4c129ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.245919 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc944be6-c533-4147-9f2d-99e3f4c129ed admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.246206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc944be6-c533-4147-9f2d-99e3f4c129ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.246485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc944be6-c533-4147-9f2d-99e3f4c129ed admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.247435 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc944be6-c533-4147-9f2d-99e3f4c129ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.249345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc944be6-c533-4147-9f2d-99e3f4c129ed admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.253815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:01.269139 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1058/2091] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:50:01.273768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc944be6-c533-4147-9f2d-99e3f4c129ed admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.286571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc944be6-c533-4147-9f2d-99e3f4c129ed admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.296247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-79e67a12-8c81-4c70-8dae-0a468535469d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac8073f8db724e8db91e9bd1ffd3c160', 'request_id': 'req-79e67a12-8c81-4c70-8dae-0a468535469d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:01.308241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-79e67a12-8c81-4c70-8dae-0a468535469d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac8073f8db724e8db91e9bd1ffd3c160', 'request_id': 'req-79e67a12-8c81-4c70-8dae-0a468535469d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:01.315226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-79e67a12-8c81-4c70-8dae-0a468535469d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac8073f8db724e8db91e9bd1ffd3c160', 'request_id': 'req-79e67a12-8c81-4c70-8dae-0a468535469d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:01.329338 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-79e67a12-8c81-4c70-8dae-0a468535469d admin admin] The token cache is being invalidate because project ac8073f8db724e8db91e9bd1ffd3c160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:01.329753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-79e67a12-8c81-4c70-8dae-0a468535469d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:01.330522 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-79e67a12-8c81-4c70-8dae-0a468535469d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:01.332706 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1034/2092] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:01 2022] DELETE /identity/v3/projects/ac8073f8db724e8db91e9bd1ffd3c160 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:50:01.337603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05069ab4-f07b-45c4-a1a3-036e57cd72a5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.337894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05069ab4-f07b-45c4-a1a3-036e57cd72a5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.338172 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05069ab4-f07b-45c4-a1a3-036e57cd72a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.365862 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-05069ab4-f07b-45c4-a1a3-036e57cd72a5 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:01.367675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-05069ab4-f07b-45c4-a1a3-036e57cd72a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:01.371642 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a194445-6282-4d6b-bece-ce132dadbaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.372607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a194445-6282-4d6b-bece-ce132dadbaf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.372896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a194445-6282-4d6b-bece-ce132dadbaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.373182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a194445-6282-4d6b-bece-ce132dadbaf2 admin admin] PATH_INFO: `/v3/projects/58dc04284f5d4e9299974ac1827943c0/users/4f0fed37f3c340609ee2b334d1eff6cd/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.380388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1035/2093] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:01.388391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17b39d-b3f1-4e1c-82f6-a5da8f638c6f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.388749 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17b39d-b3f1-4e1c-82f6-a5da8f638c6f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.389025 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17b39d-b3f1-4e1c-82f6-a5da8f638c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.390544 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a194445-6282-4d6b-bece-ce132dadbaf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58dc04284f5d4e9299974ac1827943c0, user_id=4f0fed37f3c340609ee2b334d1eff6cd, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.393340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a194445-6282-4d6b-bece-ce132dadbaf2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.409716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-fb17b39d-b3f1-4e1c-82f6-a5da8f638c6f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:01.410907 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fb17b39d-b3f1-4e1c-82f6-a5da8f638c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:01.412079 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1059/2094] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:01 2022] PUT /identity/v3/projects/58dc04284f5d4e9299974ac1827943c0/users/4f0fed37f3c340609ee2b334d1eff6cd/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:01.419359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef835ccf-4dac-4c85-acf0-bff3260da477 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.419645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef835ccf-4dac-4c85-acf0-bff3260da477 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.419917 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef835ccf-4dac-4c85-acf0-bff3260da477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.452872 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-ef835ccf-4dac-4c85-acf0-bff3260da477 None None] MFA Rules not processed for user `4f0fed37f3c340609ee2b334d1eff6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:01.454177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ef835ccf-4dac-4c85-acf0-bff3260da477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:01.516268 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1036/2095] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:50:01.519534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17b39d-b3f1-4e1c-82f6-a5da8f638c6f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.545575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17b39d-b3f1-4e1c-82f6-a5da8f638c6f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.551178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62f36e-04eb-470a-866c-14daecbb13ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.551931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62f36e-04eb-470a-866c-14daecbb13ac admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.552395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62f36e-04eb-470a-866c-14daecbb13ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.552681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62f36e-04eb-470a-866c-14daecbb13ac admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.553421 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62f36e-04eb-470a-866c-14daecbb13ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:50:01.553733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62f36e-04eb-470a-866c-14daecbb13ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.555377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc62f36e-04eb-470a-866c-14daecbb13ac admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.575544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1037/2096] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:50:01.581042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62f36e-04eb-470a-866c-14daecbb13ac admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.588563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1060/2097] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:50:01.593936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62f36e-04eb-470a-866c-14daecbb13ac admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.599092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025dd92-6054-4948-9ea1-13a93c367854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.599772 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025dd92-6054-4948-9ea1-13a93c367854 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.600076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025dd92-6054-4948-9ea1-13a93c367854 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.600358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025dd92-6054-4948-9ea1-13a93c367854 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.601681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025dd92-6054-4948-9ea1-13a93c367854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.603774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025dd92-6054-4948-9ea1-13a93c367854 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.620769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef835ccf-4dac-4c85-acf0-bff3260da477 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.622264 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ef835ccf-4dac-4c85-acf0-bff3260da477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:01.644011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1038/2098] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:01 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:01.648951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025dd92-6054-4948-9ea1-13a93c367854 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.701622 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef835ccf-4dac-4c85-acf0-bff3260da477 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.706246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:01.744210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025dd92-6054-4948-9ea1-13a93c367854 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.749944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d26272-c608-4141-ab4b-49dfc5966070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.750831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d26272-c608-4141-ab4b-49dfc5966070 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.751114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d26272-c608-4141-ab4b-49dfc5966070 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.751390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d26272-c608-4141-ab4b-49dfc5966070 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.752191 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d26272-c608-4141-ab4b-49dfc5966070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.755224 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d26272-c608-4141-ab4b-49dfc5966070 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.757850 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1b9a9-17a9-4808-8641-8f9a66f955a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.758567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1b9a9-17a9-4808-8641-8f9a66f955a7 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.758950 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1b9a9-17a9-4808-8641-8f9a66f955a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.759320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1b9a9-17a9-4808-8641-8f9a66f955a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.773905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:01.785469 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1039/2099] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:01 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:01.788668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d26272-c608-4141-ab4b-49dfc5966070 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.801684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d26272-c608-4141-ab4b-49dfc5966070 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.806461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacbc15-8740-49fc-ab7a-2b0e57fadca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.807122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacbc15-8740-49fc-ab7a-2b0e57fadca2 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.807408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacbc15-8740-49fc-ab7a-2b0e57fadca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.807685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacbc15-8740-49fc-ab7a-2b0e57fadca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.808462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacbc15-8740-49fc-ab7a-2b0e57fadca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.810083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacbc15-8740-49fc-ab7a-2b0e57fadca2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.823874 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1040/2100] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:50:01.827858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacbc15-8740-49fc-ab7a-2b0e57fadca2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.830881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1b9a9-17a9-4808-8641-8f9a66f955a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.832416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1b9a9-17a9-4808-8641-8f9a66f955a7 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.839424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacbc15-8740-49fc-ab7a-2b0e57fadca2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.844284 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1061/2101] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:50:01.844412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6b334-3bec-427b-b83f-e1d8f6c8b861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.844858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6b334-3bec-427b-b83f-e1d8f6c8b861 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.845125 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6b334-3bec-427b-b83f-e1d8f6c8b861 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.845393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6b334-3bec-427b-b83f-e1d8f6c8b861 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.846132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6b334-3bec-427b-b83f-e1d8f6c8b861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.847925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6b334-3bec-427b-b83f-e1d8f6c8b861 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.866367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1041/2102] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:50:01.867597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1b9a9-17a9-4808-8641-8f9a66f955a7 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:01.878789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1b9a9-17a9-4808-8641-8f9a66f955a7 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:01.883894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbf625-2d31-42e9-9f3e-bc7cb74e0f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:01.885117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbf625-2d31-42e9-9f3e-bc7cb74e0f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.885406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbf625-2d31-42e9-9f3e-bc7cb74e0f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.885687 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbf625-2d31-42e9-9f3e-bc7cb74e0f2d admin admin] PATH_INFO: `/v3/projects/b52b95c43de04857ab2a7813f1af8b0f/users/84c9e04fd1cd4f179a6ca4801957b288/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.908040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbf625-2d31-42e9-9f3e-bc7cb74e0f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52b95c43de04857ab2a7813f1af8b0f, user_id=84c9e04fd1cd4f179a6ca4801957b288, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:01.911028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbf625-2d31-42e9-9f3e-bc7cb74e0f2d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:01.928225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1062/2103] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:01 2022] PUT /identity/v3/projects/b52b95c43de04857ab2a7813f1af8b0f/users/84c9e04fd1cd4f179a6ca4801957b288/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:01.940318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8c95d-1e20-4cdb-968f-f65be952167d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:01.941168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8c95d-1e20-4cdb-968f-f65be952167d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:01.942441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8c95d-1e20-4cdb-968f-f65be952167d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:01.970140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dab8c95d-1e20-4cdb-968f-f65be952167d None None] MFA Rules not processed for user `84c9e04fd1cd4f179a6ca4801957b288`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:01.971371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dab8c95d-1e20-4cdb-968f-f65be952167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:02.050920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbf625-2d31-42e9-9f3e-bc7cb74e0f2d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:02.052825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-edcbf625-2d31-42e9-9f3e-bc7cb74e0f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:02.053344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1042/2104] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:50:02.058725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8c95d-1e20-4cdb-968f-f65be952167d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:02.127337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbf625-2d31-42e9-9f3e-bc7cb74e0f2d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:02.131263 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60780a4d-8c1b-439e-bf7a-0f73c742dc47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:02.132499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60780a4d-8c1b-439e-bf7a-0f73c742dc47 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:02.132843 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60780a4d-8c1b-439e-bf7a-0f73c742dc47 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:02.133136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60780a4d-8c1b-439e-bf7a-0f73c742dc47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:02.134987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-60780a4d-8c1b-439e-bf7a-0f73c742dc47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:02.152419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8c95d-1e20-4cdb-968f-f65be952167d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:02.159957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23102fcc-5d0b-4c3f-83d2-a3923bb53d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:02.161377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23102fcc-5d0b-4c3f-83d2-a3923bb53d20 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:02.162133 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23102fcc-5d0b-4c3f-83d2-a3923bb53d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:02.163254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23102fcc-5d0b-4c3f-83d2-a3923bb53d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:02.171146 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23102fcc-5d0b-4c3f-83d2-a3923bb53d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:02.175444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23102fcc-5d0b-4c3f-83d2-a3923bb53d20 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:02.196937 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:02.215436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60780a4d-8c1b-439e-bf7a-0f73c742dc47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:02.217228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60780a4d-8c1b-439e-bf7a-0f73c742dc47 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:02.218266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1043/2105] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:02 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:50:02.222840 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23102fcc-5d0b-4c3f-83d2-a3923bb53d20 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:02.230380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1063/2106] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:02.291408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:02.304829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23102fcc-5d0b-4c3f-83d2-a3923bb53d20 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:02.310739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae86a91-69e3-4e29-bc99-739cf9b0a36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:02.311556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae86a91-69e3-4e29-bc99-739cf9b0a36f admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:02.311997 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae86a91-69e3-4e29-bc99-739cf9b0a36f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:02.312363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae86a91-69e3-4e29-bc99-739cf9b0a36f admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:02.313266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae86a91-69e3-4e29-bc99-739cf9b0a36f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:02.315579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae86a91-69e3-4e29-bc99-739cf9b0a36f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:02.337612 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1044/2107] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:02 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:02.341612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60780a4d-8c1b-439e-bf7a-0f73c742dc47 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:02.358858 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60780a4d-8c1b-439e-bf7a-0f73c742dc47 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:02.363997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1bae5-ca6e-49e9-920d-c869b7bcdb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:02.364707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1bae5-ca6e-49e9-920d-c869b7bcdb85 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:02.364996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1bae5-ca6e-49e9-920d-c869b7bcdb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:02.365274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1bae5-ca6e-49e9-920d-c869b7bcdb85 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:02.366020 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1bae5-ca6e-49e9-920d-c869b7bcdb85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:02.367744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1bae5-ca6e-49e9-920d-c869b7bcdb85 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:02.381648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1064/2108] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:02.384964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae86a91-69e3-4e29-bc99-739cf9b0a36f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:02.398927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae86a91-69e3-4e29-bc99-739cf9b0a36f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:02.408617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed56e12-8b00-4a9c-b81c-c233e0ab86b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:02.413439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed56e12-8b00-4a9c-b81c-c233e0ab86b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:02.414025 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed56e12-8b00-4a9c-b81c-c233e0ab86b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:02.414025 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed56e12-8b00-4a9c-b81c-c233e0ab86b2 admin admin] PATH_INFO: `/v3/projects/aa9ced62f4344fc5aebbda8c2a6237c8/users/67d70ca831304ec39764ef5bc672e92c/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:02.442684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed56e12-8b00-4a9c-b81c-c233e0ab86b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9ced62f4344fc5aebbda8c2a6237c8, user_id=67d70ca831304ec39764ef5bc672e92c, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:02.447258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed56e12-8b00-4a9c-b81c-c233e0ab86b2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:02.464208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1045/2109] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:02 2022] PUT /identity/v3/projects/aa9ced62f4344fc5aebbda8c2a6237c8/users/67d70ca831304ec39764ef5bc672e92c/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:02.468550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1bae5-ca6e-49e9-920d-c869b7bcdb85 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:02.482644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1bae5-ca6e-49e9-920d-c869b7bcdb85 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:02.595862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0817a-04f5-4cdb-8d2d-9ad02a71031a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:02.596541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0817a-04f5-4cdb-8d2d-9ad02a71031a admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:02.596899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0817a-04f5-4cdb-8d2d-9ad02a71031a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:02.597919 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0817a-04f5-4cdb-8d2d-9ad02a71031a admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:02.598950 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0817a-04f5-4cdb-8d2d-9ad02a71031a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:02.601059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0817a-04f5-4cdb-8d2d-9ad02a71031a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:02.613313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1065/2110] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:02.619499 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed56e12-8b00-4a9c-b81c-c233e0ab86b2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:02.638467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed56e12-8b00-4a9c-b81c-c233e0ab86b2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:02.646299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8db23-761c-4165-b0d1-23aec8433ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:02.647297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8db23-761c-4165-b0d1-23aec8433ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:02.647676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8db23-761c-4165-b0d1-23aec8433ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:02.648035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8db23-761c-4165-b0d1-23aec8433ff4 admin admin] PATH_INFO: `/v3/projects/aa9ced62f4344fc5aebbda8c2a6237c8/users/67d70ca831304ec39764ef5bc672e92c/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:02.650212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea8db23-761c-4165-b0d1-23aec8433ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa9ced62f4344fc5aebbda8c2a6237c8, user_id=67d70ca831304ec39764ef5bc672e92c, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:02.653754 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea8db23-761c-4165-b0d1-23aec8433ff4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:02.673030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1046/2111] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:02 2022] PUT /identity/v3/projects/aa9ced62f4344fc5aebbda8c2a6237c8/users/67d70ca831304ec39764ef5bc672e92c/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:02.683301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63e577-e756-40f3-93f4-653ba26da412 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:02.683596 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63e577-e756-40f3-93f4-653ba26da412 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:02.683875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63e577-e756-40f3-93f4-653ba26da412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:02.711226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8a63e577-e756-40f3-93f4-653ba26da412 None None] MFA Rules not processed for user `67d70ca831304ec39764ef5bc672e92c`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:02.712484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8a63e577-e756-40f3-93f4-653ba26da412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:02.784523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1066/2112] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:02.795266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8db23-761c-4165-b0d1-23aec8433ff4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:02.852575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8db23-761c-4165-b0d1-23aec8433ff4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:02.856476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b64fd78-20e7-47f0-9515-ee9c461842c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:02.857131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64fd78-20e7-47f0-9515-ee9c461842c1 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:02.857410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64fd78-20e7-47f0-9515-ee9c461842c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:02.857679 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b64fd78-20e7-47f0-9515-ee9c461842c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:02.915281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b64fd78-20e7-47f0-9515-ee9c461842c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:02.916792 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b64fd78-20e7-47f0-9515-ee9c461842c1 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:02.925916 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1047/2113] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:03.893043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63e577-e756-40f3-93f4-653ba26da412 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:03.897395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8a63e577-e756-40f3-93f4-653ba26da412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:04.008659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:04.014139 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63e577-e756-40f3-93f4-653ba26da412 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:04.024705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:04.027860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:04.028351 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:04.028857 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] PATH_INFO: `/v3/users/aae60f23c96f46919402aed937020a79` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:04.030707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae60f23c96f46919402aed937020a79)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:04.032823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:04.111374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae60f23c96f46919402aed937020a79'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.112212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae60f23c96f46919402aed937020a79'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.124519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae60f23c96f46919402aed937020a79'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.132513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae60f23c96f46919402aed937020a79'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.145819 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae60f23c96f46919402aed937020a79'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.152625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1067/2114] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:03 2022] DELETE /identity/v3/users/aae60f23c96f46919402aed937020a79 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:04.157651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe99c5d-aac1-4000-905e-8fbbc7b76f2f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:04.158115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe99c5d-aac1-4000-905e-8fbbc7b76f2f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:04.158220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe99c5d-aac1-4000-905e-8fbbc7b76f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:04.175746 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7fe99c5d-aac1-4000-905e-8fbbc7b76f2f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:04.177009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7fe99c5d-aac1-4000-905e-8fbbc7b76f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:04.225790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:04.240729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1048/2115] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:04.253079 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:04.254545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:04.319861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076dd97-e895-443f-a566-b9527e4f7335 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:04.323984 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607659bf-fc25-44ac-a9af-7a09f6f6e753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:04.324684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607659bf-fc25-44ac-a9af-7a09f6f6e753 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:04.324975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607659bf-fc25-44ac-a9af-7a09f6f6e753 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:04.325250 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607659bf-fc25-44ac-a9af-7a09f6f6e753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:04.335598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607659bf-fc25-44ac-a9af-7a09f6f6e753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:04.337274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607659bf-fc25-44ac-a9af-7a09f6f6e753 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:04.346808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1068/2116] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:50:04.620838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe99c5d-aac1-4000-905e-8fbbc7b76f2f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:04.622686 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7fe99c5d-aac1-4000-905e-8fbbc7b76f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:04.637243 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe99c5d-aac1-4000-905e-8fbbc7b76f2f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:04.642555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:04.643596 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:04.643928 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:04.644244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] PATH_INFO: `/v3/projects/d613c89fc1dc47f0b57e9d2a0c42e78d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:04.645946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d613c89fc1dc47f0b57e9d2a0c42e78d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:04.647958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:04.697346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd613c89fc1dc47f0b57e9d2a0c42e78d', 'request_id': 'req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.705479 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd613c89fc1dc47f0b57e9d2a0c42e78d', 'request_id': 'req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.712437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd613c89fc1dc47f0b57e9d2a0c42e78d', 'request_id': 'req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.724319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] The token cache is being invalidate because project d613c89fc1dc47f0b57e9d2a0c42e78d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:04.724617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.725297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.726858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1049/2117] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:04 2022] DELETE /identity/v3/projects/d613c89fc1dc47f0b57e9d2a0c42e78d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:04.731413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607659bf-fc25-44ac-a9af-7a09f6f6e753 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:04.744664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607659bf-fc25-44ac-a9af-7a09f6f6e753 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:04.803198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:04.804256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:04.804541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:04.804815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] PATH_INFO: `/v3/users/54d68ccbc595499a97bd1fe422adcb20` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:04.806216 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54d68ccbc595499a97bd1fe422adcb20)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:04.808128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:04.832972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:04.864794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54d68ccbc595499a97bd1fe422adcb20'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.865581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54d68ccbc595499a97bd1fe422adcb20'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.876958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54d68ccbc595499a97bd1fe422adcb20'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.886791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54d68ccbc595499a97bd1fe422adcb20'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.903880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54d68ccbc595499a97bd1fe422adcb20'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:04.914466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1069/2118] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:04 2022] DELETE /identity/v3/users/54d68ccbc595499a97bd1fe422adcb20 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:05.335026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:05.336282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:05.350341 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc07b9f-1b3a-47a5-acee-950cbcfcc7f7 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:05.410820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:05.411691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:05.411968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:05.412275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] PATH_INFO: `/v3/projects/a04facd58fa9499bbf83831df037b3d4` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:05.413483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a04facd58fa9499bbf83831df037b3d4)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:05.415382 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:05.464511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a04facd58fa9499bbf83831df037b3d4', 'request_id': 'req-e7c47957-3f53-46de-9c50-b6fcee213531'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:05.474305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a04facd58fa9499bbf83831df037b3d4', 'request_id': 'req-e7c47957-3f53-46de-9c50-b6fcee213531'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:05.480178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a04facd58fa9499bbf83831df037b3d4', 'request_id': 'req-e7c47957-3f53-46de-9c50-b6fcee213531'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:05.491540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] The token cache is being invalidate because project a04facd58fa9499bbf83831df037b3d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:05.491889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:05.492586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:05.494205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1050/2119] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:05 2022] DELETE /identity/v3/projects/a04facd58fa9499bbf83831df037b3d4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:05.498533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:05.500197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:05.513820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a67c6-a7d0-46c9-ac63-3693c1812114 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:05.573925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f860cc-4b8b-4b35-a996-c0384b4158fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:05.574816 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f860cc-4b8b-4b35-a996-c0384b4158fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:05.575105 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f860cc-4b8b-4b35-a996-c0384b4158fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:05.575391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f860cc-4b8b-4b35-a996-c0384b4158fc admin admin] PATH_INFO: `/v3/users/c1396f966f51467ebb436112d538dcae` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:05.576736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f860cc-4b8b-4b35-a996-c0384b4158fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1396f966f51467ebb436112d538dcae)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:05.578705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f860cc-4b8b-4b35-a996-c0384b4158fc admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:05.628467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-86f860cc-4b8b-4b35-a996-c0384b4158fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1396f966f51467ebb436112d538dcae'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:05.629189 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-86f860cc-4b8b-4b35-a996-c0384b4158fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1396f966f51467ebb436112d538dcae'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:05.637895 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-86f860cc-4b8b-4b35-a996-c0384b4158fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1396f966f51467ebb436112d538dcae'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:05.645493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-86f860cc-4b8b-4b35-a996-c0384b4158fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1396f966f51467ebb436112d538dcae'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:05.658336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-86f860cc-4b8b-4b35-a996-c0384b4158fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1396f966f51467ebb436112d538dcae'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:05.664740 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1070/2120] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:05 2022] DELETE /identity/v3/users/c1396f966f51467ebb436112d538dcae => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:05.941439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:05.943078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:05.958719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c47957-3f53-46de-9c50-b6fcee213531 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:05.971292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:06.022053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59439c9c-7237-4772-993d-1ea3e90289a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:06.022848 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59439c9c-7237-4772-993d-1ea3e90289a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:06.023129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59439c9c-7237-4772-993d-1ea3e90289a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:06.023401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59439c9c-7237-4772-993d-1ea3e90289a9 admin admin] PATH_INFO: `/v3/projects/8b4d2c3e26b04e1d9dde9a63623c4ae8` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:06.024629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59439c9c-7237-4772-993d-1ea3e90289a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b4d2c3e26b04e1d9dde9a63623c4ae8)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:06.026561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59439c9c-7237-4772-993d-1ea3e90289a9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:06.079715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-59439c9c-7237-4772-993d-1ea3e90289a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4d2c3e26b04e1d9dde9a63623c4ae8', 'request_id': 'req-59439c9c-7237-4772-993d-1ea3e90289a9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:06.091400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-59439c9c-7237-4772-993d-1ea3e90289a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4d2c3e26b04e1d9dde9a63623c4ae8', 'request_id': 'req-59439c9c-7237-4772-993d-1ea3e90289a9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:06.100022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-59439c9c-7237-4772-993d-1ea3e90289a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4d2c3e26b04e1d9dde9a63623c4ae8', 'request_id': 'req-59439c9c-7237-4772-993d-1ea3e90289a9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:06.112570 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-59439c9c-7237-4772-993d-1ea3e90289a9 admin admin] The token cache is being invalidate because project 8b4d2c3e26b04e1d9dde9a63623c4ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:06.112860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-59439c9c-7237-4772-993d-1ea3e90289a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:06.113562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-59439c9c-7237-4772-993d-1ea3e90289a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:06.115266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1051/2121] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:05 2022] DELETE /identity/v3/projects/8b4d2c3e26b04e1d9dde9a63623c4ae8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:06.121229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6e51b-455c-4ee9-8a20-da4690bc9e22 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:06.121705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6e51b-455c-4ee9-8a20-da4690bc9e22 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:06.121789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6e51b-455c-4ee9-8a20-da4690bc9e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:06.139585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-08b6e51b-455c-4ee9-8a20-da4690bc9e22 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:06.140850 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-08b6e51b-455c-4ee9-8a20-da4690bc9e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:06.153835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:06.213755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1071/2122] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:06.222330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565d005-2f31-4584-822d-25b8dada4652 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:06.222619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565d005-2f31-4584-822d-25b8dada4652 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:06.222893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565d005-2f31-4584-822d-25b8dada4652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:06.255439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1565d005-2f31-4584-822d-25b8dada4652 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:06.256697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1565d005-2f31-4584-822d-25b8dada4652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:06.265279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1052/2123] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:06.269821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6e51b-455c-4ee9-8a20-da4690bc9e22 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:06.279907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6e51b-455c-4ee9-8a20-da4690bc9e22 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:06.285274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad071bb-20ea-41b6-945b-d97ef3402845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:06.285949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad071bb-20ea-41b6-945b-d97ef3402845 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:06.286234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad071bb-20ea-41b6-945b-d97ef3402845 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:06.286515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad071bb-20ea-41b6-945b-d97ef3402845 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:06.287247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad071bb-20ea-41b6-945b-d97ef3402845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:50:06.287721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad071bb-20ea-41b6-945b-d97ef3402845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:06.289379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad071bb-20ea-41b6-945b-d97ef3402845 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:06.304049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1072/2124] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:50:06.306568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565d005-2f31-4584-822d-25b8dada4652 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:06.316544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565d005-2f31-4584-822d-25b8dada4652 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:06.321411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19fc5a-fd56-4e18-9e93-22b06c3c44d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:06.322086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19fc5a-fd56-4e18-9e93-22b06c3c44d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:06.322370 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19fc5a-fd56-4e18-9e93-22b06c3c44d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:06.322645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19fc5a-fd56-4e18-9e93-22b06c3c44d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:06.323396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19fc5a-fd56-4e18-9e93-22b06c3c44d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:06.325406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19fc5a-fd56-4e18-9e93-22b06c3c44d9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:06.356045 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1053/2125] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:06 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:06.357849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad071bb-20ea-41b6-945b-d97ef3402845 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:06.369554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad071bb-20ea-41b6-945b-d97ef3402845 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:06.425586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c08f6-369f-4373-8b6f-e1b37693ea80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:06.426347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c08f6-369f-4373-8b6f-e1b37693ea80 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:06.426643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c08f6-369f-4373-8b6f-e1b37693ea80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:06.426962 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c08f6-369f-4373-8b6f-e1b37693ea80 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:06.427663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c08f6-369f-4373-8b6f-e1b37693ea80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:06.429704 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c08f6-369f-4373-8b6f-e1b37693ea80 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:06.450160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1073/2126] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:06 2022] POST /identity/v3/users => generated 281 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:06.454449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19fc5a-fd56-4e18-9e93-22b06c3c44d9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:06.465081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19fc5a-fd56-4e18-9e93-22b06c3c44d9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:06.470285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8c9b1-5296-4298-8f32-e5ac031b9998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:06.470949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8c9b1-5296-4298-8f32-e5ac031b9998 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:06.471226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8c9b1-5296-4298-8f32-e5ac031b9998 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:06.471630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8c9b1-5296-4298-8f32-e5ac031b9998 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:06.472432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8c9b1-5296-4298-8f32-e5ac031b9998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:06.474040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab8c9b1-5296-4298-8f32-e5ac031b9998 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:06.486169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1054/2127] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:06.490421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c08f6-369f-4373-8b6f-e1b37693ea80 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:06.501358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c08f6-369f-4373-8b6f-e1b37693ea80 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:06.507222 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86315136-9fbb-4c64-aa6a-9767079c6389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:06.507939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86315136-9fbb-4c64-aa6a-9767079c6389 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:06.508491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86315136-9fbb-4c64-aa6a-9767079c6389 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:06.508783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86315136-9fbb-4c64-aa6a-9767079c6389 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:06.509580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86315136-9fbb-4c64-aa6a-9767079c6389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:06.511282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86315136-9fbb-4c64-aa6a-9767079c6389 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:06.522860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1074/2128] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:06.528842 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8c9b1-5296-4298-8f32-e5ac031b9998 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:06.540099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8c9b1-5296-4298-8f32-e5ac031b9998 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:06.546607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765075c-d1d4-4196-bfbe-dc28063312e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:06.547499 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765075c-d1d4-4196-bfbe-dc28063312e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:06.547776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765075c-d1d4-4196-bfbe-dc28063312e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:06.548035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765075c-d1d4-4196-bfbe-dc28063312e1 admin admin] PATH_INFO: `/v3/projects/5efebefbba7c4d0ba6f28e19eca1a371/users/b65d57d5e3184f519353144a9b599ab0/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:06.563794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765075c-d1d4-4196-bfbe-dc28063312e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5efebefbba7c4d0ba6f28e19eca1a371, user_id=b65d57d5e3184f519353144a9b599ab0, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:06.566495 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765075c-d1d4-4196-bfbe-dc28063312e1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:06.582987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1055/2129] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:06 2022] PUT /identity/v3/projects/5efebefbba7c4d0ba6f28e19eca1a371/users/b65d57d5e3184f519353144a9b599ab0/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:06.594218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790084df-202e-4734-bad8-d3123b1377c0 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:06.594515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790084df-202e-4734-bad8-d3123b1377c0 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:06.594794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790084df-202e-4734-bad8-d3123b1377c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:06.621189 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-790084df-202e-4734-bad8-d3123b1377c0 None None] MFA Rules not processed for user `b65d57d5e3184f519353144a9b599ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:06.622425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-790084df-202e-4734-bad8-d3123b1377c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:06.686378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1075/2130] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:06.698244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765075c-d1d4-4196-bfbe-dc28063312e1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:06.700100 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c765075c-d1d4-4196-bfbe-dc28063312e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:06.742907 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:06.759943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765075c-d1d4-4196-bfbe-dc28063312e1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:06.764563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cdde7-a8cc-4dd0-a5f2-b075a44aa199 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:06.765252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cdde7-a8cc-4dd0-a5f2-b075a44aa199 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:06.765561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cdde7-a8cc-4dd0-a5f2-b075a44aa199 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:06.765835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cdde7-a8cc-4dd0-a5f2-b075a44aa199 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:06.776039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cdde7-a8cc-4dd0-a5f2-b075a44aa199 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:06.777531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cdde7-a8cc-4dd0-a5f2-b075a44aa199 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:06.787153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1056/2131] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:15.513310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790084df-202e-4734-bad8-d3123b1377c0 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:15.514861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-790084df-202e-4734-bad8-d3123b1377c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:15.530678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:15.586193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790084df-202e-4734-bad8-d3123b1377c0 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:15.591197 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:15.592176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:15.592460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:15.592736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] PATH_INFO: `/v3/users/077828c6ab0046258b7e0c7365f6508b` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:15.594038 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=077828c6ab0046258b7e0c7365f6508b)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:15.595992 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:15.645118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '077828c6ab0046258b7e0c7365f6508b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:15.645762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '077828c6ab0046258b7e0c7365f6508b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:15.653543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '077828c6ab0046258b7e0c7365f6508b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:15.661311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '077828c6ab0046258b7e0c7365f6508b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:15.674950 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '077828c6ab0046258b7e0c7365f6508b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:15.681308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1076/2132] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:15 2022] DELETE /identity/v3/users/077828c6ab0046258b7e0c7365f6508b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:15.685846 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d384e7-3795-4dee-9c13-0f519b925e84 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:15.686387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d384e7-3795-4dee-9c13-0f519b925e84 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:15.687178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d384e7-3795-4dee-9c13-0f519b925e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:15.705614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-81d384e7-3795-4dee-9c13-0f519b925e84 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:15.706848 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-81d384e7-3795-4dee-9c13-0f519b925e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:15.773280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1057/2133] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:15.783631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:15.785202 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:15.842573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162adcd9-404f-450f-bf3a-3529ed9ab0a3 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:15.846591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99587a16-f6b5-4bb8-a61a-a9b4352de564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:15.847244 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99587a16-f6b5-4bb8-a61a-a9b4352de564 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:15.847534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99587a16-f6b5-4bb8-a61a-a9b4352de564 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:15.847812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99587a16-f6b5-4bb8-a61a-a9b4352de564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:15.857315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99587a16-f6b5-4bb8-a61a-a9b4352de564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:15.858769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99587a16-f6b5-4bb8-a61a-a9b4352de564 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:15.869626 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1077/2134] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:16.326838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d384e7-3795-4dee-9c13-0f519b925e84 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:16.328504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-81d384e7-3795-4dee-9c13-0f519b925e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:16.347987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d384e7-3795-4dee-9c13-0f519b925e84 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:16.353616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:16.354535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:16.354906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:16.355119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] PATH_INFO: `/v3/projects/55726417bb7c4f04b274903e6b3a9c07` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:16.356486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55726417bb7c4f04b274903e6b3a9c07)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:16.358497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:16.412825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55726417bb7c4f04b274903e6b3a9c07', 'request_id': 'req-2d4e0328-aa1e-49f1-ab95-176061228b95'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:16.421475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55726417bb7c4f04b274903e6b3a9c07', 'request_id': 'req-2d4e0328-aa1e-49f1-ab95-176061228b95'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:16.427159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55726417bb7c4f04b274903e6b3a9c07', 'request_id': 'req-2d4e0328-aa1e-49f1-ab95-176061228b95'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:16.438370 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] The token cache is being invalidate because project 55726417bb7c4f04b274903e6b3a9c07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:16.438661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:16.439291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:16.440880 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1058/2135] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:16 2022] DELETE /identity/v3/projects/55726417bb7c4f04b274903e6b3a9c07 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:16.445087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99587a16-f6b5-4bb8-a61a-a9b4352de564 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:16.454710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99587a16-f6b5-4bb8-a61a-a9b4352de564 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:16.487037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:16.515414 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:16.516334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:16.516619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:16.516894 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] PATH_INFO: `/v3/users/22d565f5221648c293152a609d7a9879` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:16.518072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] RBAC: Authorizing `identity:delete_user(user_id=22d565f5221648c293152a609d7a9879)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:16.520467 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:16.570815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22d565f5221648c293152a609d7a9879'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:16.571473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22d565f5221648c293152a609d7a9879'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:16.580207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22d565f5221648c293152a609d7a9879'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:16.588946 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22d565f5221648c293152a609d7a9879'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:16.600879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22d565f5221648c293152a609d7a9879'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:16.607354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1078/2136] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:16 2022] DELETE /identity/v3/users/22d565f5221648c293152a609d7a9879 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:17.089476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:17.090806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:17.120680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:17.122581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:17.168860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e0328-aa1e-49f1-ab95-176061228b95 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:17.191414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:17.220251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf6bda-acdd-4bb2-95fc-8960dcabbdce admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:17.225708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873c08f-dbdf-4506-9469-aca51fe83696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:17.227135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873c08f-dbdf-4506-9469-aca51fe83696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:17.227639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873c08f-dbdf-4506-9469-aca51fe83696 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:17.227939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873c08f-dbdf-4506-9469-aca51fe83696 admin admin] PATH_INFO: `/v3/projects/d877015ea5834afa9aabd4b681c50bd3` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:17.229718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873c08f-dbdf-4506-9469-aca51fe83696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d877015ea5834afa9aabd4b681c50bd3)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:17.232737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873c08f-dbdf-4506-9469-aca51fe83696 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:17.236781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c4894-ab50-47a5-8e4b-a14522fbb3f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:17.238219 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c4894-ab50-47a5-8e4b-a14522fbb3f5 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:17.238667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c4894-ab50-47a5-8e4b-a14522fbb3f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:17.238986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c4894-ab50-47a5-8e4b-a14522fbb3f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:17.242106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f68c4894-ab50-47a5-8e4b-a14522fbb3f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:17.288886 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6873c08f-dbdf-4506-9469-aca51fe83696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd877015ea5834afa9aabd4b681c50bd3', 'request_id': 'req-6873c08f-dbdf-4506-9469-aca51fe83696'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:17.299356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6873c08f-dbdf-4506-9469-aca51fe83696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd877015ea5834afa9aabd4b681c50bd3', 'request_id': 'req-6873c08f-dbdf-4506-9469-aca51fe83696'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:17.300920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:17.312369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6873c08f-dbdf-4506-9469-aca51fe83696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd877015ea5834afa9aabd4b681c50bd3', 'request_id': 'req-6873c08f-dbdf-4506-9469-aca51fe83696'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:17.327221 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6873c08f-dbdf-4506-9469-aca51fe83696 admin admin] The token cache is being invalidate because project d877015ea5834afa9aabd4b681c50bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:17.327221 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6873c08f-dbdf-4506-9469-aca51fe83696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:17.328209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6873c08f-dbdf-4506-9469-aca51fe83696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:17.332046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1079/2137] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:17 2022] DELETE /identity/v3/projects/d877015ea5834afa9aabd4b681c50bd3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:50:17.334693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c4894-ab50-47a5-8e4b-a14522fbb3f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:17.336363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80ab10-c3c8-45df-bdb5-c8de2437a496 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:17.336698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80ab10-c3c8-45df-bdb5-c8de2437a496 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:17.337075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80ab10-c3c8-45df-bdb5-c8de2437a496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:17.337191 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c4894-ab50-47a5-8e4b-a14522fbb3f5 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:17.348941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1059/2138] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:17.355482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-6e80ab10-c3c8-45df-bdb5-c8de2437a496 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:17.356822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6e80ab10-c3c8-45df-bdb5-c8de2437a496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:17.429952 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1080/2139] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:50:17.431664 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827204b-a9bc-4bdb-b567-bba9aa8e8708 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:17.432278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827204b-a9bc-4bdb-b567-bba9aa8e8708 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:17.433094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827204b-a9bc-4bdb-b567-bba9aa8e8708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:17.465071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a827204b-a9bc-4bdb-b567-bba9aa8e8708 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:17.466504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a827204b-a9bc-4bdb-b567-bba9aa8e8708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:17.476870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1060/2140] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:17.481971 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80ab10-c3c8-45df-bdb5-c8de2437a496 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:17.494205 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80ab10-c3c8-45df-bdb5-c8de2437a496 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:17.508324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1432f2-6a32-4cac-aaf5-a3b58951a6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:17.509754 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1432f2-6a32-4cac-aaf5-a3b58951a6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:17.512857 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1432f2-6a32-4cac-aaf5-a3b58951a6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:17.513274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1432f2-6a32-4cac-aaf5-a3b58951a6c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:17.514420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1432f2-6a32-4cac-aaf5-a3b58951a6c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:50:17.515169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1432f2-6a32-4cac-aaf5-a3b58951a6c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:17.518278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1432f2-6a32-4cac-aaf5-a3b58951a6c2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:17.537311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1081/2141] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:50:17.539462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827204b-a9bc-4bdb-b567-bba9aa8e8708 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:17.553773 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827204b-a9bc-4bdb-b567-bba9aa8e8708 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:17.559026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367a475-f0ac-4c2b-8522-3084bfc94f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:17.559711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367a475-f0ac-4c2b-8522-3084bfc94f0b admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:17.559982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367a475-f0ac-4c2b-8522-3084bfc94f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:17.560310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367a475-f0ac-4c2b-8522-3084bfc94f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:17.561007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367a475-f0ac-4c2b-8522-3084bfc94f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:17.562941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367a475-f0ac-4c2b-8522-3084bfc94f0b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:17.595543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1061/2142] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:17 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:17.596958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1432f2-6a32-4cac-aaf5-a3b58951a6c2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:17.607748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1432f2-6a32-4cac-aaf5-a3b58951a6c2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:17.677021 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80152119-b815-4e54-a774-391fad011fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:17.677844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80152119-b815-4e54-a774-391fad011fee admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:17.678154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80152119-b815-4e54-a774-391fad011fee admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:17.678416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80152119-b815-4e54-a774-391fad011fee admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:17.679108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80152119-b815-4e54-a774-391fad011fee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:17.681110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80152119-b815-4e54-a774-391fad011fee admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:17.707807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1082/2143] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:17 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:17.712479 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367a475-f0ac-4c2b-8522-3084bfc94f0b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:17.724423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367a475-f0ac-4c2b-8522-3084bfc94f0b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:17.739436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be8b4f-2946-468b-b639-c1e35d94cca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:17.740202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be8b4f-2946-468b-b639-c1e35d94cca2 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:17.740566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be8b4f-2946-468b-b639-c1e35d94cca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:17.740744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be8b4f-2946-468b-b639-c1e35d94cca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:17.741481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be8b4f-2946-468b-b639-c1e35d94cca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:17.743102 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be8b4f-2946-468b-b639-c1e35d94cca2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:17.761052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1062/2144] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:17.763080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80152119-b815-4e54-a774-391fad011fee admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:17.776345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80152119-b815-4e54-a774-391fad011fee admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:17.781201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962d06d-736d-436b-904d-bf6834435fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:17.781896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962d06d-736d-436b-904d-bf6834435fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:17.782136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962d06d-736d-436b-904d-bf6834435fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:17.782416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962d06d-736d-436b-904d-bf6834435fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:17.783367 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962d06d-736d-436b-904d-bf6834435fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:17.784952 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962d06d-736d-436b-904d-bf6834435fb1 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:17.801884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1083/2145] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:17.802818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be8b4f-2946-468b-b639-c1e35d94cca2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:17.814965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be8b4f-2946-468b-b639-c1e35d94cca2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:17.820566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4abc7-2068-47b4-9a01-d21f906b9fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:17.821583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4abc7-2068-47b4-9a01-d21f906b9fca admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:17.822712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4abc7-2068-47b4-9a01-d21f906b9fca admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:17.822712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4abc7-2068-47b4-9a01-d21f906b9fca admin admin] PATH_INFO: `/v3/projects/64b5c22a5a7447718164af38b437c405/users/2be338cf0ed24d8095b71980e42e5fac/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:17.841848 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4abc7-2068-47b4-9a01-d21f906b9fca admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b5c22a5a7447718164af38b437c405, user_id=2be338cf0ed24d8095b71980e42e5fac, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:17.844836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4abc7-2068-47b4-9a01-d21f906b9fca admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:17.859769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1063/2146] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:17 2022] PUT /identity/v3/projects/64b5c22a5a7447718164af38b437c405/users/2be338cf0ed24d8095b71980e42e5fac/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:50:17.870107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20507d1b-c4ff-4c02-b667-7eadf58f08d3 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:17.870403 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20507d1b-c4ff-4c02-b667-7eadf58f08d3 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:17.870685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20507d1b-c4ff-4c02-b667-7eadf58f08d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:17.897453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-20507d1b-c4ff-4c02-b667-7eadf58f08d3 None None] MFA Rules not processed for user `2be338cf0ed24d8095b71980e42e5fac`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:17.898662 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-20507d1b-c4ff-4c02-b667-7eadf58f08d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:17.939535 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:17.966207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1084/2147] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:17.977669 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4abc7-2068-47b4-9a01-d21f906b9fca admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:17.979474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-98e4abc7-2068-47b4-9a01-d21f906b9fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:18.059297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:18.064731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4abc7-2068-47b4-9a01-d21f906b9fca admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:18.069255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc159f-381d-4e7e-ad03-5af64211a2dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7297224dc73c4358b60f1b096c0fb5bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:18.069943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc159f-381d-4e7e-ad03-5af64211a2dd service glance] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:18.070221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc159f-381d-4e7e-ad03-5af64211a2dd service glance] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:18.070497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc159f-381d-4e7e-ad03-5af64211a2dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:18.079267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc159f-381d-4e7e-ad03-5af64211a2dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:18.080750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc159f-381d-4e7e-ad03-5af64211a2dd service glance] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:18.089799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1064/2148] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:23.655965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20507d1b-c4ff-4c02-b667-7eadf58f08d3 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:23.657542 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-20507d1b-c4ff-4c02-b667-7eadf58f08d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:23.726306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20507d1b-c4ff-4c02-b667-7eadf58f08d3 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:23.730988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:23.731918 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:23.732264 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:23.732501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] PATH_INFO: `/v3/users/84c9e04fd1cd4f179a6ca4801957b288` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:23.733733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c9e04fd1cd4f179a6ca4801957b288)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:23.735686 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:23.783441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c9e04fd1cd4f179a6ca4801957b288'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:23.784154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c9e04fd1cd4f179a6ca4801957b288'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:23.793300 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c9e04fd1cd4f179a6ca4801957b288'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:23.800446 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c9e04fd1cd4f179a6ca4801957b288'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:23.812861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c9e04fd1cd4f179a6ca4801957b288'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:23.819214 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1085/2149] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:23 2022] DELETE /identity/v3/users/84c9e04fd1cd4f179a6ca4801957b288 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:23.823845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa062f-2d05-4b05-9e35-63bec1fa10e5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:23.824399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa062f-2d05-4b05-9e35-63bec1fa10e5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:23.824399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa062f-2d05-4b05-9e35-63bec1fa10e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:23.843187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d2fa062f-2d05-4b05-9e35-63bec1fa10e5 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:23.844438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d2fa062f-2d05-4b05-9e35-63bec1fa10e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:23.908444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1065/2150] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:23.918491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:23.919952 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:23.978094 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74141b14-a38b-426b-a035-f75505c5cedf admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:23.982190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfb49d-9c2f-4199-a6c9-63d95b3ef956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:23.982839 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfb49d-9c2f-4199-a6c9-63d95b3ef956 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:23.983119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfb49d-9c2f-4199-a6c9-63d95b3ef956 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:23.983393 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfb49d-9c2f-4199-a6c9-63d95b3ef956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:23.992813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfb49d-9c2f-4199-a6c9-63d95b3ef956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:23.994242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfb49d-9c2f-4199-a6c9-63d95b3ef956 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:24.003566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1086/2151] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:50:24.295673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa062f-2d05-4b05-9e35-63bec1fa10e5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:24.299410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d2fa062f-2d05-4b05-9e35-63bec1fa10e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:24.321573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa062f-2d05-4b05-9e35-63bec1fa10e5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:24.327267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:24.328263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:24.328657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:24.328894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] PATH_INFO: `/v3/projects/b52b95c43de04857ab2a7813f1af8b0f` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:24.330477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52b95c43de04857ab2a7813f1af8b0f)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:24.333021 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:24.408689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52b95c43de04857ab2a7813f1af8b0f', 'request_id': 'req-e5883379-aa0e-49e3-8c76-6fd50d016316'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:24.418797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52b95c43de04857ab2a7813f1af8b0f', 'request_id': 'req-e5883379-aa0e-49e3-8c76-6fd50d016316'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:24.424333 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52b95c43de04857ab2a7813f1af8b0f', 'request_id': 'req-e5883379-aa0e-49e3-8c76-6fd50d016316'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:24.436866 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] The token cache is being invalidate because project b52b95c43de04857ab2a7813f1af8b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:24.437302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:24.437954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:24.439496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1066/2152] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:24 2022] DELETE /identity/v3/projects/b52b95c43de04857ab2a7813f1af8b0f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:24.443587 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfb49d-9c2f-4199-a6c9-63d95b3ef956 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:24.448518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:24.455402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfb49d-9c2f-4199-a6c9-63d95b3ef956 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:24.545735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e44d72-d6e1-4016-8728-1099e4b87ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:24.547257 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e44d72-d6e1-4016-8728-1099e4b87ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:24.547733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e44d72-d6e1-4016-8728-1099e4b87ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:24.550808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e44d72-d6e1-4016-8728-1099e4b87ba6 admin admin] PATH_INFO: `/v3/users/67d70ca831304ec39764ef5bc672e92c` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:24.552100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e44d72-d6e1-4016-8728-1099e4b87ba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d70ca831304ec39764ef5bc672e92c)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:24.555496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e44d72-d6e1-4016-8728-1099e4b87ba6 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:24.626507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d7e44d72-d6e1-4016-8728-1099e4b87ba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d70ca831304ec39764ef5bc672e92c'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:24.627156 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d7e44d72-d6e1-4016-8728-1099e4b87ba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d70ca831304ec39764ef5bc672e92c'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:24.636908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d7e44d72-d6e1-4016-8728-1099e4b87ba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d70ca831304ec39764ef5bc672e92c'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:24.646047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d7e44d72-d6e1-4016-8728-1099e4b87ba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d70ca831304ec39764ef5bc672e92c'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:24.660231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-d7e44d72-d6e1-4016-8728-1099e4b87ba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d70ca831304ec39764ef5bc672e92c'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:24.666949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1087/2153] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:24 2022] DELETE /identity/v3/users/67d70ca831304ec39764ef5bc672e92c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:25.025735 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:25.027546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:25.041871 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5883379-aa0e-49e3-8c76-6fd50d016316 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:25.106962 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc75de-9f71-40f7-8faa-e09029cd83cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:25.107899 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc75de-9f71-40f7-8faa-e09029cd83cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:25.108221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc75de-9f71-40f7-8faa-e09029cd83cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:25.108516 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc75de-9f71-40f7-8faa-e09029cd83cf admin admin] PATH_INFO: `/v3/projects/aa9ced62f4344fc5aebbda8c2a6237c8` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:25.109756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc75de-9f71-40f7-8faa-e09029cd83cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa9ced62f4344fc5aebbda8c2a6237c8)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:25.111747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc75de-9f71-40f7-8faa-e09029cd83cf admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:25.129661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:25.163322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4acc75de-9f71-40f7-8faa-e09029cd83cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa9ced62f4344fc5aebbda8c2a6237c8', 'request_id': 'req-4acc75de-9f71-40f7-8faa-e09029cd83cf'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:25.173703 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4acc75de-9f71-40f7-8faa-e09029cd83cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa9ced62f4344fc5aebbda8c2a6237c8', 'request_id': 'req-4acc75de-9f71-40f7-8faa-e09029cd83cf'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:25.178807 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4acc75de-9f71-40f7-8faa-e09029cd83cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa9ced62f4344fc5aebbda8c2a6237c8', 'request_id': 'req-4acc75de-9f71-40f7-8faa-e09029cd83cf'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:25.190064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4acc75de-9f71-40f7-8faa-e09029cd83cf admin admin] The token cache is being invalidate because project aa9ced62f4344fc5aebbda8c2a6237c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:25.190347 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4acc75de-9f71-40f7-8faa-e09029cd83cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:25.191002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-4acc75de-9f71-40f7-8faa-e09029cd83cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:25.192651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1067/2154] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:25 2022] DELETE /identity/v3/projects/aa9ced62f4344fc5aebbda8c2a6237c8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:25.199533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f35f9-2eea-4046-885c-d08878a76cba None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:25.199759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f35f9-2eea-4046-885c-d08878a76cba None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:25.200078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f35f9-2eea-4046-885c-d08878a76cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:25.220437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-dd3f35f9-2eea-4046-885c-d08878a76cba None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:25.221765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-dd3f35f9-2eea-4046-885c-d08878a76cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:25.282106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1088/2155] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:25.292357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c4c18-4d49-4352-99a3-387da6b36034 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:25.292683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c4c18-4d49-4352-99a3-387da6b36034 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:25.292983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c4c18-4d49-4352-99a3-387da6b36034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:25.315996 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e34c4c18-4d49-4352-99a3-387da6b36034 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:25.317362 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e34c4c18-4d49-4352-99a3-387da6b36034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:25.325762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1068/2156] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:25.330751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f35f9-2eea-4046-885c-d08878a76cba None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:25.341970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f35f9-2eea-4046-885c-d08878a76cba None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:25.346963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa0daf-4fca-4423-b694-2875fa509692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:25.347631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa0daf-4fca-4423-b694-2875fa509692 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:25.347941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa0daf-4fca-4423-b694-2875fa509692 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:25.348253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa0daf-4fca-4423-b694-2875fa509692 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:25.349027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa0daf-4fca-4423-b694-2875fa509692 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:50:25.349342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa0daf-4fca-4423-b694-2875fa509692 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:25.350973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa0daf-4fca-4423-b694-2875fa509692 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:25.362667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1089/2157] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:50:25.367348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c4c18-4d49-4352-99a3-387da6b36034 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:25.376611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c4c18-4d49-4352-99a3-387da6b36034 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:25.381905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44ee3d-73c2-47fd-b3fb-0c197ff97c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:25.382564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44ee3d-73c2-47fd-b3fb-0c197ff97c6b admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:25.382848 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44ee3d-73c2-47fd-b3fb-0c197ff97c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:25.383121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f44ee3d-73c2-47fd-b3fb-0c197ff97c6b admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:25.383811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44ee3d-73c2-47fd-b3fb-0c197ff97c6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:25.385744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f44ee3d-73c2-47fd-b3fb-0c197ff97c6b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:25.414817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1069/2158] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:25 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:25.418936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa0daf-4fca-4423-b694-2875fa509692 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:25.429065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa0daf-4fca-4423-b694-2875fa509692 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:25.433851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:25.496510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d716-47a8-4466-a53f-c42a738c6a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:25.497212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d716-47a8-4466-a53f-c42a738c6a74 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:25.497479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d716-47a8-4466-a53f-c42a738c6a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:25.497781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d716-47a8-4466-a53f-c42a738c6a74 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:25.498425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39d716-47a8-4466-a53f-c42a738c6a74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:25.500461 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39d716-47a8-4466-a53f-c42a738c6a74 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:25.520639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1090/2159] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:25 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:25.525057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44ee3d-73c2-47fd-b3fb-0c197ff97c6b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:25.538355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f44ee3d-73c2-47fd-b3fb-0c197ff97c6b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:25.543520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3ace9-ace6-4ec2-afcb-8130efa8cb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:25.545588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3ace9-ace6-4ec2-afcb-8130efa8cb82 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:25.545922 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3ace9-ace6-4ec2-afcb-8130efa8cb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:25.546177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3ace9-ace6-4ec2-afcb-8130efa8cb82 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:25.546888 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3ace9-ace6-4ec2-afcb-8130efa8cb82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:25.548689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3ace9-ace6-4ec2-afcb-8130efa8cb82 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:25.564744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1070/2160] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:25.565957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d716-47a8-4466-a53f-c42a738c6a74 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:25.577143 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d716-47a8-4466-a53f-c42a738c6a74 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:25.583768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3293c9b-340f-4d56-8219-7bda725518c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:25.584450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3293c9b-340f-4d56-8219-7bda725518c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:25.584728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3293c9b-340f-4d56-8219-7bda725518c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:25.585368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3293c9b-340f-4d56-8219-7bda725518c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:25.586104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3293c9b-340f-4d56-8219-7bda725518c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:25.587918 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3293c9b-340f-4d56-8219-7bda725518c4 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:25.601936 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1091/2161] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:25.606973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3ace9-ace6-4ec2-afcb-8130efa8cb82 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:25.620627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3ace9-ace6-4ec2-afcb-8130efa8cb82 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:25.628808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8583a-27c5-4eb2-a37c-93de937ede0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:25.631324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8583a-27c5-4eb2-a37c-93de937ede0f admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:25.631610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8583a-27c5-4eb2-a37c-93de937ede0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:25.631883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8583a-27c5-4eb2-a37c-93de937ede0f admin admin] PATH_INFO: `/v3/projects/894d4374f1e7417bbdb6fb4bc81e1ad9/users/dfabe031d63543279c5faecebdaa30e9/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:25.651266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8583a-27c5-4eb2-a37c-93de937ede0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=894d4374f1e7417bbdb6fb4bc81e1ad9, user_id=dfabe031d63543279c5faecebdaa30e9, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:25.651563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8583a-27c5-4eb2-a37c-93de937ede0f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:25.676344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1071/2162] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:25 2022] PUT /identity/v3/projects/894d4374f1e7417bbdb6fb4bc81e1ad9/users/dfabe031d63543279c5faecebdaa30e9/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:25.678820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5b80c-21ee-4696-bb71-746c48eb57e1 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:25.679286 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5b80c-21ee-4696-bb71-746c48eb57e1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:25.679374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5b80c-21ee-4696-bb71-746c48eb57e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:25.708075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-83d5b80c-21ee-4696-bb71-746c48eb57e1 None None] MFA Rules not processed for user `dfabe031d63543279c5faecebdaa30e9`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:25.709407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-83d5b80c-21ee-4696-bb71-746c48eb57e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:25.776520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1092/2163] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:50:25.781800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8583a-27c5-4eb2-a37c-93de937ede0f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:25.847570 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8583a-27c5-4eb2-a37c-93de937ede0f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:25.852575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf03714-5718-4649-9bdf-31c332087772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:25.853244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf03714-5718-4649-9bdf-31c332087772 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:25.853520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf03714-5718-4649-9bdf-31c332087772 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:25.853794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf03714-5718-4649-9bdf-31c332087772 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:25.854486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf03714-5718-4649-9bdf-31c332087772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:25.856444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf03714-5718-4649-9bdf-31c332087772 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:25.886255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1072/2164] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:25 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:25.887413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5b80c-21ee-4696-bb71-746c48eb57e1 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:25.899790 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5b80c-21ee-4696-bb71-746c48eb57e1 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:25.967991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6a9af-ee56-4de1-9995-fc4461fac53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:25.968779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6a9af-ee56-4de1-9995-fc4461fac53d admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:25.969215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6a9af-ee56-4de1-9995-fc4461fac53d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:25.969568 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6a9af-ee56-4de1-9995-fc4461fac53d admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:25.970404 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6a9af-ee56-4de1-9995-fc4461fac53d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:25.972454 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6a9af-ee56-4de1-9995-fc4461fac53d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:25.996280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1093/2165] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:25 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:50:25.997536 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf03714-5718-4649-9bdf-31c332087772 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:26.009898 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf03714-5718-4649-9bdf-31c332087772 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:26.016985 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840a07f-34b9-401d-a28e-3cb8cfcdddba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:26.017691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840a07f-34b9-401d-a28e-3cb8cfcdddba admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:26.017976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840a07f-34b9-401d-a28e-3cb8cfcdddba admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:26.018289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840a07f-34b9-401d-a28e-3cb8cfcdddba admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:26.019393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840a07f-34b9-401d-a28e-3cb8cfcdddba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:26.021057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840a07f-34b9-401d-a28e-3cb8cfcdddba admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:26.032740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1073/2166] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:26.036869 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6a9af-ee56-4de1-9995-fc4461fac53d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:26.047194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6a9af-ee56-4de1-9995-fc4461fac53d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:26.051955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af37892-2f74-4030-ae47-49d020b4be9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:26.052659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af37892-2f74-4030-ae47-49d020b4be9f admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:26.052948 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af37892-2f74-4030-ae47-49d020b4be9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:26.053231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af37892-2f74-4030-ae47-49d020b4be9f admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:26.053965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af37892-2f74-4030-ae47-49d020b4be9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:26.055590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af37892-2f74-4030-ae47-49d020b4be9f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:26.056966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:26.068273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1094/2167] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:26.071813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840a07f-34b9-401d-a28e-3cb8cfcdddba admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:26.082238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840a07f-34b9-401d-a28e-3cb8cfcdddba admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:26.086873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0b1ed-5990-4410-a5d4-ec539b63e2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:26.087786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0b1ed-5990-4410-a5d4-ec539b63e2d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:26.088105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0b1ed-5990-4410-a5d4-ec539b63e2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:26.088379 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0b1ed-5990-4410-a5d4-ec539b63e2d9 admin admin] PATH_INFO: `/v3/projects/fdcc3d2e023a4c24a40846b3fd9734de/users/33d27d9f79db4020a2eae1785947b3d6/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:26.099721 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:26.104527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0b1ed-5990-4410-a5d4-ec539b63e2d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdcc3d2e023a4c24a40846b3fd9734de, user_id=33d27d9f79db4020a2eae1785947b3d6, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:26.107726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0b1ed-5990-4410-a5d4-ec539b63e2d9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:26.123391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1074/2168] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:26 2022] PUT /identity/v3/projects/fdcc3d2e023a4c24a40846b3fd9734de/users/33d27d9f79db4020a2eae1785947b3d6/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:26.134015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb34474-a236-4491-affb-9b7869b00968 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:26.134318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb34474-a236-4491-affb-9b7869b00968 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:26.134584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb34474-a236-4491-affb-9b7869b00968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:26.163138 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8cb34474-a236-4491-affb-9b7869b00968 None None] MFA Rules not processed for user `33d27d9f79db4020a2eae1785947b3d6`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:26.164420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8cb34474-a236-4491-affb-9b7869b00968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:26.238356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1095/2169] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:26.244563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0b1ed-5990-4410-a5d4-ec539b63e2d9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:26.309438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0b1ed-5990-4410-a5d4-ec539b63e2d9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:26.314505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863cf7f-f1b9-4b26-bc86-0106c3a88751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:26.315175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863cf7f-f1b9-4b26-bc86-0106c3a88751 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:26.315465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863cf7f-f1b9-4b26-bc86-0106c3a88751 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:26.315741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863cf7f-f1b9-4b26-bc86-0106c3a88751 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:26.316488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863cf7f-f1b9-4b26-bc86-0106c3a88751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:26.318562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863cf7f-f1b9-4b26-bc86-0106c3a88751 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:26.346972 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1075/2170] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:26 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:26.351855 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb34474-a236-4491-affb-9b7869b00968 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:26.365070 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb34474-a236-4491-affb-9b7869b00968 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:26.426432 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfdb4f-fc04-46e3-9c25-3716cce2f89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:26.427166 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfdb4f-fc04-46e3-9c25-3716cce2f89d admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:26.427453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfdb4f-fc04-46e3-9c25-3716cce2f89d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:26.427731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfdb4f-fc04-46e3-9c25-3716cce2f89d admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:26.428486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfdb4f-fc04-46e3-9c25-3716cce2f89d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:26.430470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfdb4f-fc04-46e3-9c25-3716cce2f89d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:26.455492 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1096/2171] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:26 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:26.456769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863cf7f-f1b9-4b26-bc86-0106c3a88751 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:26.467213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863cf7f-f1b9-4b26-bc86-0106c3a88751 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:26.471851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ab72e-17a0-4dd7-89a8-998821bc095a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:26.472538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ab72e-17a0-4dd7-89a8-998821bc095a admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:26.472823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ab72e-17a0-4dd7-89a8-998821bc095a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:26.473102 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ab72e-17a0-4dd7-89a8-998821bc095a admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:26.473890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ab72e-17a0-4dd7-89a8-998821bc095a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:26.475524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ab72e-17a0-4dd7-89a8-998821bc095a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:26.486460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1076/2172] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:26.491315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfdb4f-fc04-46e3-9c25-3716cce2f89d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:26.502601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfdb4f-fc04-46e3-9c25-3716cce2f89d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:26.507718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f8b72-9e2c-4918-8fc2-de08c977e853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:26.508699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f8b72-9e2c-4918-8fc2-de08c977e853 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:26.509029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f8b72-9e2c-4918-8fc2-de08c977e853 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:26.509830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f8b72-9e2c-4918-8fc2-de08c977e853 admin admin] PATH_INFO: `/v3/projects/1b32087ea85a4f6d9bdc5a260c353229/users/1b4a9f0a559b44b89f8e5220393b40a4/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:26.526265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f8b72-9e2c-4918-8fc2-de08c977e853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b32087ea85a4f6d9bdc5a260c353229, user_id=1b4a9f0a559b44b89f8e5220393b40a4, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:26.529021 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f8b72-9e2c-4918-8fc2-de08c977e853 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:26.547146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1097/2173] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:26 2022] PUT /identity/v3/projects/1b32087ea85a4f6d9bdc5a260c353229/users/1b4a9f0a559b44b89f8e5220393b40a4/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:26.551649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ab72e-17a0-4dd7-89a8-998821bc095a admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:26.565722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ab72e-17a0-4dd7-89a8-998821bc095a admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:26.633507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b887045-7761-436e-a454-b18d5c0863d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:26.634153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b887045-7761-436e-a454-b18d5c0863d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:26.634482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b887045-7761-436e-a454-b18d5c0863d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:26.634714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b887045-7761-436e-a454-b18d5c0863d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:26.635508 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b887045-7761-436e-a454-b18d5c0863d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:26.637084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b887045-7761-436e-a454-b18d5c0863d3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:26.651130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1077/2174] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:26.655262 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f8b72-9e2c-4918-8fc2-de08c977e853 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:26.666735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f8b72-9e2c-4918-8fc2-de08c977e853 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:26.672709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970d864-f0d6-493e-b6e8-397bb0bd5364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:26.673735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970d864-f0d6-493e-b6e8-397bb0bd5364 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:26.673997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970d864-f0d6-493e-b6e8-397bb0bd5364 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:26.674277 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970d864-f0d6-493e-b6e8-397bb0bd5364 admin admin] PATH_INFO: `/v3/projects/1b32087ea85a4f6d9bdc5a260c353229/users/1b4a9f0a559b44b89f8e5220393b40a4/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:26.676409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970d864-f0d6-493e-b6e8-397bb0bd5364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b32087ea85a4f6d9bdc5a260c353229, user_id=1b4a9f0a559b44b89f8e5220393b40a4, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:26.679350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970d864-f0d6-493e-b6e8-397bb0bd5364 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:26.693897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1098/2175] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:26 2022] PUT /identity/v3/projects/1b32087ea85a4f6d9bdc5a260c353229/users/1b4a9f0a559b44b89f8e5220393b40a4/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:26.705506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35368a0a-b5c3-46b6-8b2d-58a560332ce0 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:26.705799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35368a0a-b5c3-46b6-8b2d-58a560332ce0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:26.706081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35368a0a-b5c3-46b6-8b2d-58a560332ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:26.733790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-35368a0a-b5c3-46b6-8b2d-58a560332ce0 None None] MFA Rules not processed for user `1b4a9f0a559b44b89f8e5220393b40a4`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:26.735131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-35368a0a-b5c3-46b6-8b2d-58a560332ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:26.789529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:26.820197 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1078/2176] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:26.832475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970d864-f0d6-493e-b6e8-397bb0bd5364 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:26.834419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c970d864-f0d6-493e-b6e8-397bb0bd5364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:26.879531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:26.896147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970d864-f0d6-493e-b6e8-397bb0bd5364 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:26.899908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb36fcc-d6b7-4f2c-8cfc-6032b98884b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:26.900591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb36fcc-d6b7-4f2c-8cfc-6032b98884b1 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:26.900878 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb36fcc-d6b7-4f2c-8cfc-6032b98884b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:26.901172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb36fcc-d6b7-4f2c-8cfc-6032b98884b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:26.965166 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb36fcc-d6b7-4f2c-8cfc-6032b98884b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:26.966751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb36fcc-d6b7-4f2c-8cfc-6032b98884b1 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:26.975951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1099/2177] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:36.815964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=7 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:36.971774 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=31 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:40.111745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35368a0a-b5c3-46b6-8b2d-58a560332ce0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:40.123267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35368a0a-b5c3-46b6-8b2d-58a560332ce0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:40.126832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a45844-2de1-4534-9239-ddff3799bbf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:40.127488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a45844-2de1-4534-9239-ddff3799bbf5 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:40.127772 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a45844-2de1-4534-9239-ddff3799bbf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:40.128058 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a45844-2de1-4534-9239-ddff3799bbf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:40.186662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a45844-2de1-4534-9239-ddff3799bbf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:40.188134 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a45844-2de1-4534-9239-ddff3799bbf5 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:40.198094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1079/2178] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:40.801365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb36fcc-d6b7-4f2c-8cfc-6032b98884b1 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:40.810700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb36fcc-d6b7-4f2c-8cfc-6032b98884b1 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:40.815120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbecd94-1ff0-4ce1-b015-3311ffe81335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:40.815882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbecd94-1ff0-4ce1-b015-3311ffe81335 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:40.816243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbecd94-1ff0-4ce1-b015-3311ffe81335 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:40.816569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbecd94-1ff0-4ce1-b015-3311ffe81335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:40.827319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbecd94-1ff0-4ce1-b015-3311ffe81335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:40.829052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbecd94-1ff0-4ce1-b015-3311ffe81335 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:40.840690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1100/2179] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:45.044939 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a45844-2de1-4534-9239-ddff3799bbf5 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:45.047027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-69a45844-2de1-4534-9239-ddff3799bbf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:45.126337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a45844-2de1-4534-9239-ddff3799bbf5 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:45.131563 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:45.132527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:45.132815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:45.133094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] PATH_INFO: `/v3/users/7c767bec3f4c4c9c8698afce8e40fd19` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:45.134369 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c767bec3f4c4c9c8698afce8e40fd19)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:45.136830 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:45.186279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c767bec3f4c4c9c8698afce8e40fd19'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.197797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c767bec3f4c4c9c8698afce8e40fd19'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.198495 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c767bec3f4c4c9c8698afce8e40fd19'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.209410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c767bec3f4c4c9c8698afce8e40fd19'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.224682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c767bec3f4c4c9c8698afce8e40fd19'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.234859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1080/2180] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:45 2022] DELETE /identity/v3/users/7c767bec3f4c4c9c8698afce8e40fd19 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:45.432833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbecd94-1ff0-4ce1-b015-3311ffe81335 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:45.445316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbecd94-1ff0-4ce1-b015-3311ffe81335 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:45.541438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:45.542099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:45.542393 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:45.542673 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] PATH_INFO: `/v3/projects/e0fd252826f243cd9430ba1a4cee95c5` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:45.543873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0fd252826f243cd9430ba1a4cee95c5)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:45.545832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:45.631873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0fd252826f243cd9430ba1a4cee95c5', 'request_id': 'req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.641171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0fd252826f243cd9430ba1a4cee95c5', 'request_id': 'req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.650397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0fd252826f243cd9430ba1a4cee95c5', 'request_id': 'req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.664296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] The token cache is being invalidate because project e0fd252826f243cd9430ba1a4cee95c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:45.664594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.665259 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.666965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1101/2181] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:45 2022] DELETE /identity/v3/projects/e0fd252826f243cd9430ba1a4cee95c5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:45.671126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:45.673485 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:45.690028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5a2f0-d472-45bc-a139-776affb0bcdf admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:45.704601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:45.772447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:45.773867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:45.774929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:45.775220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] PATH_INFO: `/v3/users/9d9d4b27802846fe9669b938b6a9e215` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:45.776720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d9d4b27802846fe9669b938b6a9e215)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:45.778740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:45.838753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d9d4b27802846fe9669b938b6a9e215'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.856283 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d9d4b27802846fe9669b938b6a9e215'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.856283 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d9d4b27802846fe9669b938b6a9e215'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.866410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d9d4b27802846fe9669b938b6a9e215'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.883549 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d9d4b27802846fe9669b938b6a9e215'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:45.891217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1081/2182] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:45 2022] DELETE /identity/v3/users/9d9d4b27802846fe9669b938b6a9e215 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:46.239045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:46.240693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:46.249023 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:46.250857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:46.278892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b1c15-e8e4-4ca5-8522-47c5fc7f0748 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:46.338395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5a8d7-8a9f-4b01-8378-9c15b81c1f97 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:46.345240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326aca0-1203-4798-aa0c-ef5587b378cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:46.346575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326aca0-1203-4798-aa0c-ef5587b378cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:46.346923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326aca0-1203-4798-aa0c-ef5587b378cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:46.347519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326aca0-1203-4798-aa0c-ef5587b378cc admin admin] PATH_INFO: `/v3/users/4f0fed37f3c340609ee2b334d1eff6cd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:46.347642 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e6caf-786a-48b1-8e3d-2a3653c99008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:46.348506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e6caf-786a-48b1-8e3d-2a3653c99008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:46.348787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e6caf-786a-48b1-8e3d-2a3653c99008 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:46.349062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e6caf-786a-48b1-8e3d-2a3653c99008 admin admin] PATH_INFO: `/v3/projects/aee2b9c236af4831a343a10c84445ee7` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:46.357275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326aca0-1203-4798-aa0c-ef5587b378cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f0fed37f3c340609ee2b334d1eff6cd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:46.357700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e6caf-786a-48b1-8e3d-2a3653c99008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aee2b9c236af4831a343a10c84445ee7)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:46.359486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326aca0-1203-4798-aa0c-ef5587b378cc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:46.360365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e6caf-786a-48b1-8e3d-2a3653c99008 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:46.389287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:46.417212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:46.427454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6326aca0-1203-4798-aa0c-ef5587b378cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f0fed37f3c340609ee2b334d1eff6cd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:46.428244 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f45e6caf-786a-48b1-8e3d-2a3653c99008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aee2b9c236af4831a343a10c84445ee7', 'request_id': 'req-f45e6caf-786a-48b1-8e3d-2a3653c99008'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:46.439164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6326aca0-1203-4798-aa0c-ef5587b378cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f0fed37f3c340609ee2b334d1eff6cd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:46.440567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6326aca0-1203-4798-aa0c-ef5587b378cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f0fed37f3c340609ee2b334d1eff6cd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:46.445305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f45e6caf-786a-48b1-8e3d-2a3653c99008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aee2b9c236af4831a343a10c84445ee7', 'request_id': 'req-f45e6caf-786a-48b1-8e3d-2a3653c99008'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:46.449343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6326aca0-1203-4798-aa0c-ef5587b378cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f0fed37f3c340609ee2b334d1eff6cd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:46.461047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f45e6caf-786a-48b1-8e3d-2a3653c99008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aee2b9c236af4831a343a10c84445ee7', 'request_id': 'req-f45e6caf-786a-48b1-8e3d-2a3653c99008'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:46.463719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-6326aca0-1203-4798-aa0c-ef5587b378cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f0fed37f3c340609ee2b334d1eff6cd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:46.471251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1082/2183] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:46 2022] DELETE /identity/v3/users/4f0fed37f3c340609ee2b334d1eff6cd => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:46.475862 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57e9ec-ec61-4b13-a190-29c120425fe1 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:46.476269 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57e9ec-ec61-4b13-a190-29c120425fe1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:46.476618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57e9ec-ec61-4b13-a190-29c120425fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:46.478545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f45e6caf-786a-48b1-8e3d-2a3653c99008 admin admin] The token cache is being invalidate because project aee2b9c236af4831a343a10c84445ee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:46.478937 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f45e6caf-786a-48b1-8e3d-2a3653c99008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:46.479938 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f45e6caf-786a-48b1-8e3d-2a3653c99008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:46.482430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1102/2184] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:46 2022] DELETE /identity/v3/projects/aee2b9c236af4831a343a10c84445ee7 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:50:46.486917 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637abb91-e146-4ded-8fdc-7c9fa7cf8137 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:46.487372 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637abb91-e146-4ded-8fdc-7c9fa7cf8137 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:46.487806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637abb91-e146-4ded-8fdc-7c9fa7cf8137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:46.496319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6b57e9ec-ec61-4b13-a190-29c120425fe1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:46.497524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6b57e9ec-ec61-4b13-a190-29c120425fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:46.514671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-637abb91-e146-4ded-8fdc-7c9fa7cf8137 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:46.516110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-637abb91-e146-4ded-8fdc-7c9fa7cf8137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:46.567238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1083/2185] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:50:46.579685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57e9ec-ec61-4b13-a190-29c120425fe1 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:46.581433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6b57e9ec-ec61-4b13-a190-29c120425fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:46.584092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1103/2186] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:46.592248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f1bc8-1e7e-461b-ad5a-d56e73966ad1 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:46.592543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f1bc8-1e7e-461b-ad5a-d56e73966ad1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:46.592825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f1bc8-1e7e-461b-ad5a-d56e73966ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:46.612253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-969f1bc8-1e7e-461b-ad5a-d56e73966ad1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:46.613464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-969f1bc8-1e7e-461b-ad5a-d56e73966ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:46.623316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1104/2187] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:46.633574 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f1bc8-1e7e-461b-ad5a-d56e73966ad1 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:46.646750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f1bc8-1e7e-461b-ad5a-d56e73966ad1 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:46.651212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57e9ec-ec61-4b13-a190-29c120425fe1 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:46.652418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae37fa-d012-41fc-943d-cd2605b021a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:46.656787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae37fa-d012-41fc-943d-cd2605b021a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:46.656787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae37fa-d012-41fc-943d-cd2605b021a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:46.656787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae37fa-d012-41fc-943d-cd2605b021a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:46.656787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae37fa-d012-41fc-943d-cd2605b021a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:50:46.656787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae37fa-d012-41fc-943d-cd2605b021a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:46.657160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70590bcd-2509-4029-9f10-80a56e11e9bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:46.657160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70590bcd-2509-4029-9f10-80a56e11e9bf service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:46.657160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70590bcd-2509-4029-9f10-80a56e11e9bf service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:46.657160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70590bcd-2509-4029-9f10-80a56e11e9bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:46.657428 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae37fa-d012-41fc-943d-cd2605b021a2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:46.669866 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70590bcd-2509-4029-9f10-80a56e11e9bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:46.670708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1105/2188] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:50:46.671621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70590bcd-2509-4029-9f10-80a56e11e9bf service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:46.675496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae37fa-d012-41fc-943d-cd2605b021a2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:46.684055 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1084/2189] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:50:46.698142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae37fa-d012-41fc-943d-cd2605b021a2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:46.704082 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98fdb5-d666-4d60-8e77-9e51e4d18adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:46.704770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98fdb5-d666-4d60-8e77-9e51e4d18adc admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:46.705058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98fdb5-d666-4d60-8e77-9e51e4d18adc admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:46.705514 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98fdb5-d666-4d60-8e77-9e51e4d18adc admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:46.706216 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98fdb5-d666-4d60-8e77-9e51e4d18adc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:46.708425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98fdb5-d666-4d60-8e77-9e51e4d18adc admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:46.748044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1106/2190] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:46 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:50:46.751693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70590bcd-2509-4029-9f10-80a56e11e9bf service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:46.762424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70590bcd-2509-4029-9f10-80a56e11e9bf service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:46.834249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827f5cd-ee2c-412e-b461-9d22bb33db25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:46.835460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827f5cd-ee2c-412e-b461-9d22bb33db25 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:46.835460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827f5cd-ee2c-412e-b461-9d22bb33db25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:46.835777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827f5cd-ee2c-412e-b461-9d22bb33db25 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:46.836776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827f5cd-ee2c-412e-b461-9d22bb33db25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:46.838782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827f5cd-ee2c-412e-b461-9d22bb33db25 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:46.846078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:46.877094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1085/2191] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:46 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:46.878058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98fdb5-d666-4d60-8e77-9e51e4d18adc admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:46.890907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98fdb5-d666-4d60-8e77-9e51e4d18adc admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:46.897491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e6eea-2044-429b-9dea-f92d9b9335a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:46.898481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e6eea-2044-429b-9dea-f92d9b9335a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:46.898931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e6eea-2044-429b-9dea-f92d9b9335a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:46.899211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e6eea-2044-429b-9dea-f92d9b9335a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:46.900048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e6eea-2044-429b-9dea-f92d9b9335a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:46.901686 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e6eea-2044-429b-9dea-f92d9b9335a2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:46.913504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1107/2192] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:46.917653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827f5cd-ee2c-412e-b461-9d22bb33db25 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:46.930737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827f5cd-ee2c-412e-b461-9d22bb33db25 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:46.935937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802fbb80-57a1-414b-9db5-222a9c4f0759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:46.936661 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fbb80-57a1-414b-9db5-222a9c4f0759 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:46.936952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fbb80-57a1-414b-9db5-222a9c4f0759 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:46.937232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802fbb80-57a1-414b-9db5-222a9c4f0759 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:46.937972 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802fbb80-57a1-414b-9db5-222a9c4f0759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:46.944796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802fbb80-57a1-414b-9db5-222a9c4f0759 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:46.951155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1086/2193] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:50:46.955407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e6eea-2044-429b-9dea-f92d9b9335a2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:46.968602 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e6eea-2044-429b-9dea-f92d9b9335a2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:46.974034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d9f5-ee7c-4474-be69-cdb02a15a3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:46.974993 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d9f5-ee7c-4474-be69-cdb02a15a3d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:46.975277 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d9f5-ee7c-4474-be69-cdb02a15a3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:46.975551 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752d9f5-ee7c-4474-be69-cdb02a15a3d9 admin admin] PATH_INFO: `/v3/projects/e7afaabd6e4d4ea3aebc76e7d86576ab/users/c07f8303c56741b9a70e82144a57b796/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:46.991691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752d9f5-ee7c-4474-be69-cdb02a15a3d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7afaabd6e4d4ea3aebc76e7d86576ab, user_id=c07f8303c56741b9a70e82144a57b796, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:46.994531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752d9f5-ee7c-4474-be69-cdb02a15a3d9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:47.008717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1108/2194] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:46 2022] PUT /identity/v3/projects/e7afaabd6e4d4ea3aebc76e7d86576ab/users/c07f8303c56741b9a70e82144a57b796/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:47.021506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda0eac-c269-4903-a55c-c79877146c7a None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.021902 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda0eac-c269-4903-a55c-c79877146c7a None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.022185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda0eac-c269-4903-a55c-c79877146c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.034437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d9f5-ee7c-4474-be69-cdb02a15a3d9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:47.038739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6752d9f5-ee7c-4474-be69-cdb02a15a3d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:47.074863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9fda0eac-c269-4903-a55c-c79877146c7a None None] MFA Rules not processed for user `c07f8303c56741b9a70e82144a57b796`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:47.076764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9fda0eac-c269-4903-a55c-c79877146c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:47.122849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:47.172019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752d9f5-ee7c-4474-be69-cdb02a15a3d9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:47.177723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:47.180051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.180051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.180051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] PATH_INFO: `/v3/projects/58dc04284f5d4e9299974ac1827943c0` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.181747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58dc04284f5d4e9299974ac1827943c0)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:47.183738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:47.185518 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1087/2195] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:47.189384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda0eac-c269-4903-a55c-c79877146c7a None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:47.190854 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9fda0eac-c269-4903-a55c-c79877146c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:47.218399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda0eac-c269-4903-a55c-c79877146c7a None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:47.226193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8ff46-39b5-479c-b84d-fe02c81e5c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:47.227400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8ff46-39b5-479c-b84d-fe02c81e5c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.228572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8ff46-39b5-479c-b84d-fe02c81e5c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.229175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8ff46-39b5-479c-b84d-fe02c81e5c5c admin admin] PATH_INFO: `/v3/users/dfabe031d63543279c5faecebdaa30e9` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.230580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8ff46-39b5-479c-b84d-fe02c81e5c5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfabe031d63543279c5faecebdaa30e9)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:47.232975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8ff46-39b5-479c-b84d-fe02c81e5c5c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:47.261764 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58dc04284f5d4e9299974ac1827943c0', 'request_id': 'req-6020d6a7-4df4-4fea-910b-abe7dbb291f5'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:47.271441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58dc04284f5d4e9299974ac1827943c0', 'request_id': 'req-6020d6a7-4df4-4fea-910b-abe7dbb291f5'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:47.278794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58dc04284f5d4e9299974ac1827943c0', 'request_id': 'req-6020d6a7-4df4-4fea-910b-abe7dbb291f5'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:47.289537 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] The token cache is being invalidate because project 58dc04284f5d4e9299974ac1827943c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:47.289716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d9e8ff46-39b5-479c-b84d-fe02c81e5c5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfabe031d63543279c5faecebdaa30e9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:47.290175 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:47.290633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:47.292448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1109/2196] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:47 2022] DELETE /identity/v3/projects/58dc04284f5d4e9299974ac1827943c0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:50:47.293142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:47.296884 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:47.306400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d9e8ff46-39b5-479c-b84d-fe02c81e5c5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfabe031d63543279c5faecebdaa30e9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:47.307186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d9e8ff46-39b5-479c-b84d-fe02c81e5c5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfabe031d63543279c5faecebdaa30e9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:47.317205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d9e8ff46-39b5-479c-b84d-fe02c81e5c5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfabe031d63543279c5faecebdaa30e9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:47.338763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d9e8ff46-39b5-479c-b84d-fe02c81e5c5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfabe031d63543279c5faecebdaa30e9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:47.351773 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1088/2197] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:47 2022] DELETE /identity/v3/users/dfabe031d63543279c5faecebdaa30e9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:47.358697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7087e-ce6a-4b0d-bc3b-a435cb03b24e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.359001 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7087e-ce6a-4b0d-bc3b-a435cb03b24e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.359332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7087e-ce6a-4b0d-bc3b-a435cb03b24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.373139 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020d6a7-4df4-4fea-910b-abe7dbb291f5 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:47.380704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-70a7087e-ce6a-4b0d-bc3b-a435cb03b24e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:47.381911 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-70a7087e-ce6a-4b0d-bc3b-a435cb03b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:47.429866 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:47.448511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf00e8-31f4-4dcf-bb93-ade67ac75526 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:47.449226 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf00e8-31f4-4dcf-bb93-ade67ac75526 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.449523 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf00e8-31f4-4dcf-bb93-ade67ac75526 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.449783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf00e8-31f4-4dcf-bb93-ade67ac75526 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.451734 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b8bf00e8-31f4-4dcf-bb93-ade67ac75526 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:47.472104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1089/2198] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:47.472104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5e09e-b685-4999-ac49-daa8ce8d5a24 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.472104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5e09e-b685-4999-ac49-daa8ce8d5a24 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.472104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5e09e-b685-4999-ac49-daa8ce8d5a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.489410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-42b5e09e-b685-4999-ac49-daa8ce8d5a24 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:47.490547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-42b5e09e-b685-4999-ac49-daa8ce8d5a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:47.497724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:47.498788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1090/2199] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:47.500829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895dac0-a534-4fb7-96b9-4b1163268443 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.501119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895dac0-a534-4fb7-96b9-4b1163268443 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.501584 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895dac0-a534-4fb7-96b9-4b1163268443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.527671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b895dac0-a534-4fb7-96b9-4b1163268443 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:47.528954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b895dac0-a534-4fb7-96b9-4b1163268443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:47.538252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1091/2200] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:47.539166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895dac0-a534-4fb7-96b9-4b1163268443 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:47.540687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b895dac0-a534-4fb7-96b9-4b1163268443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:47.551634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf00e8-31f4-4dcf-bb93-ade67ac75526 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:47.553306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf00e8-31f4-4dcf-bb93-ade67ac75526 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:47.564708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1110/2201] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:47.565470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf00e8-31f4-4dcf-bb93-ade67ac75526 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:47.576200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf00e8-31f4-4dcf-bb93-ade67ac75526 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:47.581192 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d104b-512c-4a37-95fa-2282f284db77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:47.581863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d104b-512c-4a37-95fa-2282f284db77 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.582152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d104b-512c-4a37-95fa-2282f284db77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.582433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d104b-512c-4a37-95fa-2282f284db77 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.583522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d104b-512c-4a37-95fa-2282f284db77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:50:47.583747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d104b-512c-4a37-95fa-2282f284db77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:47.585648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d104b-512c-4a37-95fa-2282f284db77 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:47.599322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1111/2202] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:50:47.603844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d104b-512c-4a37-95fa-2282f284db77 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:47.614753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d104b-512c-4a37-95fa-2282f284db77 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:47.619218 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895dac0-a534-4fb7-96b9-4b1163268443 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:47.621091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a6e84-51a4-44a2-867e-f666048c170a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:47.621867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a6e84-51a4-44a2-867e-f666048c170a admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.622535 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a6e84-51a4-44a2-867e-f666048c170a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.622890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a6e84-51a4-44a2-867e-f666048c170a admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.623419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36c546-52bc-4641-8834-c3505a5226bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:47.624082 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a6e84-51a4-44a2-867e-f666048c170a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:47.624253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36c546-52bc-4641-8834-c3505a5226bd service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.624557 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36c546-52bc-4641-8834-c3505a5226bd service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.624836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36c546-52bc-4641-8834-c3505a5226bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.626460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a6e84-51a4-44a2-867e-f666048c170a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:47.636462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36c546-52bc-4641-8834-c3505a5226bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:47.637940 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36c546-52bc-4641-8834-c3505a5226bd service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:47.660477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1092/2203] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:47.677519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1112/2204] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:47 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:50:47.684177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36c546-52bc-4641-8834-c3505a5226bd service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:47.705553 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36c546-52bc-4641-8834-c3505a5226bd service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:47.812354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee397e59-879b-4430-bca3-e6514c7a99a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:47.813050 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee397e59-879b-4430-bca3-e6514c7a99a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.813332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee397e59-879b-4430-bca3-e6514c7a99a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.813772 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee397e59-879b-4430-bca3-e6514c7a99a5 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.814649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee397e59-879b-4430-bca3-e6514c7a99a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:47.816967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee397e59-879b-4430-bca3-e6514c7a99a5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:47.853622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1093/2205] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:47 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:47.858220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a6e84-51a4-44a2-867e-f666048c170a admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:47.874814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a6e84-51a4-44a2-867e-f666048c170a admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:47.880565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86a91d-79ca-4438-866e-adea0b8be171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:47.881434 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86a91d-79ca-4438-866e-adea0b8be171 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.881829 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86a91d-79ca-4438-866e-adea0b8be171 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.882163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86a91d-79ca-4438-866e-adea0b8be171 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.882962 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86a91d-79ca-4438-866e-adea0b8be171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:47.885295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86a91d-79ca-4438-866e-adea0b8be171 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:47.902436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1113/2206] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:47.903454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee397e59-879b-4430-bca3-e6514c7a99a5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:47.914594 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee397e59-879b-4430-bca3-e6514c7a99a5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:47.920270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0e066-f0c8-4af3-a642-b14487905f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:47.920965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0e066-f0c8-4af3-a642-b14487905f4c admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.921254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0e066-f0c8-4af3-a642-b14487905f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.921533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0e066-f0c8-4af3-a642-b14487905f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:47.922272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0e066-f0c8-4af3-a642-b14487905f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:47.924902 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0e066-f0c8-4af3-a642-b14487905f4c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:47.948285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1094/2207] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:47.955190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86a91d-79ca-4438-866e-adea0b8be171 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:47.970902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86a91d-79ca-4438-866e-adea0b8be171 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:47.983733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47174c9-2455-420d-b66e-b0d682dc8b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:47.984789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47174c9-2455-420d-b66e-b0d682dc8b77 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:47.985117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47174c9-2455-420d-b66e-b0d682dc8b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:47.985396 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47174c9-2455-420d-b66e-b0d682dc8b77 admin admin] PATH_INFO: `/v3/projects/fe0e4ddd0d164c66962a1de9e9261c3e/users/d3ec9afe54cf479888a6af7eb3a7dd72/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:48.023304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47174c9-2455-420d-b66e-b0d682dc8b77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe0e4ddd0d164c66962a1de9e9261c3e, user_id=d3ec9afe54cf479888a6af7eb3a7dd72, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:48.033572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47174c9-2455-420d-b66e-b0d682dc8b77 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:48.062713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1114/2208] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:47 2022] PUT /identity/v3/projects/fe0e4ddd0d164c66962a1de9e9261c3e/users/d3ec9afe54cf479888a6af7eb3a7dd72/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:48.064860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9662a6-28c3-4799-94a2-05d7e1de6a09 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:48.065168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9662a6-28c3-4799-94a2-05d7e1de6a09 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:48.065520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9662a6-28c3-4799-94a2-05d7e1de6a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:48.094746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47174c9-2455-420d-b66e-b0d682dc8b77 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:48.096833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c47174c9-2455-420d-b66e-b0d682dc8b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:48.105106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9d9662a6-28c3-4799-94a2-05d7e1de6a09 None None] MFA Rules not processed for user `d3ec9afe54cf479888a6af7eb3a7dd72`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:48.106827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9d9662a6-28c3-4799-94a2-05d7e1de6a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:48.185571 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:48.216954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47174c9-2455-420d-b66e-b0d682dc8b77 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:48.222035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:48.222930 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:48.223206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:48.223481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] PATH_INFO: `/v3/projects/894d4374f1e7417bbdb6fb4bc81e1ad9` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:48.224849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=894d4374f1e7417bbdb6fb4bc81e1ad9)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:48.226877 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:48.249541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1095/2209] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:50:48.255746 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9662a6-28c3-4799-94a2-05d7e1de6a09 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:48.269363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9662a6-28c3-4799-94a2-05d7e1de6a09 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:48.275132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326cc61-dd55-4192-b93e-52cc813a2ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:48.275881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326cc61-dd55-4192-b93e-52cc813a2ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:48.276397 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326cc61-dd55-4192-b93e-52cc813a2ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:48.276676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326cc61-dd55-4192-b93e-52cc813a2ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:48.277407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326cc61-dd55-4192-b93e-52cc813a2ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:48.279431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326cc61-dd55-4192-b93e-52cc813a2ef4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:48.304294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '894d4374f1e7417bbdb6fb4bc81e1ad9', 'request_id': 'req-6de92476-6b5f-4ed2-8764-e3453e693df1'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:48.317663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '894d4374f1e7417bbdb6fb4bc81e1ad9', 'request_id': 'req-6de92476-6b5f-4ed2-8764-e3453e693df1'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:48.327026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1096/2210] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:48 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:48.328210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326cc61-dd55-4192-b93e-52cc813a2ef4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:48.331632 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '894d4374f1e7417bbdb6fb4bc81e1ad9', 'request_id': 'req-6de92476-6b5f-4ed2-8764-e3453e693df1'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:48.335466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:48.347067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] The token cache is being invalidate because project 894d4374f1e7417bbdb6fb4bc81e1ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:48.347798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:48.348612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:48.351313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1115/2211] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:48 2022] DELETE /identity/v3/projects/894d4374f1e7417bbdb6fb4bc81e1ad9 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:48.355044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:48.356507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:48.417758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326cc61-dd55-4192-b93e-52cc813a2ef4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:48.423502 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cd5c3-e924-48fb-b144-2e7b823df267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:48.424365 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cd5c3-e924-48fb-b144-2e7b823df267 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:48.424752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cd5c3-e924-48fb-b144-2e7b823df267 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:48.425126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cd5c3-e924-48fb-b144-2e7b823df267 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:48.425970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169cd5c3-e924-48fb-b144-2e7b823df267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:48.428099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169cd5c3-e924-48fb-b144-2e7b823df267 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:48.460454 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de92476-6b5f-4ed2-8764-e3453e693df1 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:48.461121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1097/2212] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:48 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:48.466158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cd5c3-e924-48fb-b144-2e7b823df267 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:48.468191 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-169cd5c3-e924-48fb-b144-2e7b823df267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:48.470455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ae570-9b30-46cf-a7b0-25f296864fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:48.471982 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ae570-9b30-46cf-a7b0-25f296864fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:48.472443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ae570-9b30-46cf-a7b0-25f296864fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:48.473055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ae570-9b30-46cf-a7b0-25f296864fdb admin admin] PATH_INFO: `/v3/users/33d27d9f79db4020a2eae1785947b3d6` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:48.478988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ae570-9b30-46cf-a7b0-25f296864fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d27d9f79db4020a2eae1785947b3d6)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:48.479387 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ae570-9b30-46cf-a7b0-25f296864fdb admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:48.490540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cd5c3-e924-48fb-b144-2e7b823df267 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:48.496386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7410228-d34f-495d-ba9c-b2d11c57c1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:48.497298 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7410228-d34f-495d-ba9c-b2d11c57c1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:48.497729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7410228-d34f-495d-ba9c-b2d11c57c1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:48.498392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7410228-d34f-495d-ba9c-b2d11c57c1e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:48.498986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7410228-d34f-495d-ba9c-b2d11c57c1e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:48.500845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7410228-d34f-495d-ba9c-b2d11c57c1e5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:48.512040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1098/2213] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:50:48.515867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7410228-d34f-495d-ba9c-b2d11c57c1e5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:48.526418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7410228-d34f-495d-ba9c-b2d11c57c1e5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:48.533301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0709bbf-4def-4e81-9c0f-a88873f08e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:48.541607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0709bbf-4def-4e81-9c0f-a88873f08e31 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:48.541607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0709bbf-4def-4e81-9c0f-a88873f08e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:48.541607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0709bbf-4def-4e81-9c0f-a88873f08e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:48.541607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0709bbf-4def-4e81-9c0f-a88873f08e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:48.542042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0709bbf-4def-4e81-9c0f-a88873f08e31 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:48.550023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-506ae570-9b30-46cf-a7b0-25f296864fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d27d9f79db4020a2eae1785947b3d6'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:48.550773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-506ae570-9b30-46cf-a7b0-25f296864fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d27d9f79db4020a2eae1785947b3d6'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:48.558990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1099/2214] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:50:48.559860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0709bbf-4def-4e81-9c0f-a88873f08e31 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:48.560903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-506ae570-9b30-46cf-a7b0-25f296864fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d27d9f79db4020a2eae1785947b3d6'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:48.561875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d0709bbf-4def-4e81-9c0f-a88873f08e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:48.571136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-506ae570-9b30-46cf-a7b0-25f296864fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d27d9f79db4020a2eae1785947b3d6'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:48.578713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0709bbf-4def-4e81-9c0f-a88873f08e31 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:48.585031 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-506ae570-9b30-46cf-a7b0-25f296864fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d27d9f79db4020a2eae1785947b3d6'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:48.592491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0eaff-04e0-4199-94ec-035791d70afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:48.593689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1116/2215] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:48 2022] DELETE /identity/v3/users/33d27d9f79db4020a2eae1785947b3d6 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:50:48.596487 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0eaff-04e0-4199-94ec-035791d70afe admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:48.596835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0eaff-04e0-4199-94ec-035791d70afe admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:48.597062 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0eaff-04e0-4199-94ec-035791d70afe admin admin] PATH_INFO: `/v3/projects/cb218c3cf4ba48b1bf5e154f525f1a35/users/c3fa169998114da19e4dcc8dec432150/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:48.614552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0eaff-04e0-4199-94ec-035791d70afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb218c3cf4ba48b1bf5e154f525f1a35, user_id=c3fa169998114da19e4dcc8dec432150, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:48.618424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0eaff-04e0-4199-94ec-035791d70afe admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:48.636500 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1100/2216] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:48 2022] PUT /identity/v3/projects/cb218c3cf4ba48b1bf5e154f525f1a35/users/c3fa169998114da19e4dcc8dec432150/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:48.649791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1bda0-ce6b-48ce-84da-f18ea7bb7e04 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:48.650113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1bda0-ce6b-48ce-84da-f18ea7bb7e04 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:48.650369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1bda0-ce6b-48ce-84da-f18ea7bb7e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:48.691182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-3de1bda0-ce6b-48ce-84da-f18ea7bb7e04 None None] MFA Rules not processed for user `c3fa169998114da19e4dcc8dec432150`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:48.692605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3de1bda0-ce6b-48ce-84da-f18ea7bb7e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:48.788867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:48.803066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1117/2217] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:48.804475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0eaff-04e0-4199-94ec-035791d70afe admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:48.898002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0eaff-04e0-4199-94ec-035791d70afe admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:48.903079 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49303eb7-934d-4806-85a0-c11ef022a216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:48.903984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49303eb7-934d-4806-85a0-c11ef022a216 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:48.904143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49303eb7-934d-4806-85a0-c11ef022a216 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:48.904363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49303eb7-934d-4806-85a0-c11ef022a216 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:48.905062 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49303eb7-934d-4806-85a0-c11ef022a216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:48.906986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49303eb7-934d-4806-85a0-c11ef022a216 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:48.911063 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:48.953424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1101/2218] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:48 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:48.955470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1bda0-ce6b-48ce-84da-f18ea7bb7e04 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:48.978401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1bda0-ce6b-48ce-84da-f18ea7bb7e04 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:49.011872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49303eb7-934d-4806-85a0-c11ef022a216 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:49.013854 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-49303eb7-934d-4806-85a0-c11ef022a216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:49.092281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea104f-cd28-4b5f-aafb-aaa3a1f6f268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:49.093227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea104f-cd28-4b5f-aafb-aaa3a1f6f268 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:49.093516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea104f-cd28-4b5f-aafb-aaa3a1f6f268 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:49.093793 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea104f-cd28-4b5f-aafb-aaa3a1f6f268 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:49.094729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea104f-cd28-4b5f-aafb-aaa3a1f6f268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:49.104961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea104f-cd28-4b5f-aafb-aaa3a1f6f268 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:49.132427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1118/2219] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:48 2022] POST /identity/v3/users => generated 295 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:49.133811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea104f-cd28-4b5f-aafb-aaa3a1f6f268 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:49.148746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea104f-cd28-4b5f-aafb-aaa3a1f6f268 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:49.150799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49303eb7-934d-4806-85a0-c11ef022a216 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:49.155150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf2700-6da6-48a0-bc89-c334ef1509e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:49.156549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf2700-6da6-48a0-bc89-c334ef1509e9 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:49.156911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf2700-6da6-48a0-bc89-c334ef1509e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:49.157549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf2700-6da6-48a0-bc89-c334ef1509e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:49.157722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:49.158318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf2700-6da6-48a0-bc89-c334ef1509e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:49.158954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:49.159380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:49.159989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf2700-6da6-48a0-bc89-c334ef1509e9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:49.160493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] PATH_INFO: `/v3/projects/fdcc3d2e023a4c24a40846b3fd9734de` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:49.162252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdcc3d2e023a4c24a40846b3fd9734de)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:49.165936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:49.181215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1119/2220] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:49.182542 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf2700-6da6-48a0-bc89-c334ef1509e9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:49.195419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf2700-6da6-48a0-bc89-c334ef1509e9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:49.201823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f1226-6899-4641-83c7-367a3b34e493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:49.202840 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f1226-6899-4641-83c7-367a3b34e493 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:49.203198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f1226-6899-4641-83c7-367a3b34e493 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:49.203631 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f1226-6899-4641-83c7-367a3b34e493 admin admin] PATH_INFO: `/v3/projects/09cf401afc3a45218b219083d0e31100/users/f07c9f83691449dd8686a68434d8f257/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:49.240733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f1226-6899-4641-83c7-367a3b34e493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cf401afc3a45218b219083d0e31100, user_id=f07c9f83691449dd8686a68434d8f257, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:49.245084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f1226-6899-4641-83c7-367a3b34e493 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:49.258939 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdcc3d2e023a4c24a40846b3fd9734de', 'request_id': 'req-73e306bb-c532-421c-bf16-079e52816dcd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:49.276816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdcc3d2e023a4c24a40846b3fd9734de', 'request_id': 'req-73e306bb-c532-421c-bf16-079e52816dcd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:49.277060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1120/2221] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:49 2022] PUT /identity/v3/projects/09cf401afc3a45218b219083d0e31100/users/f07c9f83691449dd8686a68434d8f257/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:49.285534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdcc3d2e023a4c24a40846b3fd9734de', 'request_id': 'req-73e306bb-c532-421c-bf16-079e52816dcd'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:49.287907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f1226-6899-4641-83c7-367a3b34e493 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:49.314435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] The token cache is being invalidate because project fdcc3d2e023a4c24a40846b3fd9734de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:49.314734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:49.315683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:49.321665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1102/2222] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:49 2022] DELETE /identity/v3/projects/fdcc3d2e023a4c24a40846b3fd9734de => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:49.322663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:49.327042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:49.410426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f1226-6899-4641-83c7-367a3b34e493 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:49.420182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c822f2-1414-4ab6-a371-24f325e661d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:49.420879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c822f2-1414-4ab6-a371-24f325e661d9 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:49.421159 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c822f2-1414-4ab6-a371-24f325e661d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:49.421431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c822f2-1414-4ab6-a371-24f325e661d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:49.422169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c822f2-1414-4ab6-a371-24f325e661d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:49.425065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c822f2-1414-4ab6-a371-24f325e661d9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:49.455896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e306bb-c532-421c-bf16-079e52816dcd admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:49.460641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1121/2223] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:49.461885 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c822f2-1414-4ab6-a371-24f325e661d9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:49.464453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a2c822f2-1414-4ab6-a371-24f325e661d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:49.466523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:49.467697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:49.467971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:49.468762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] PATH_INFO: `/v3/users/1b4a9f0a559b44b89f8e5220393b40a4` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:49.471647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4a9f0a559b44b89f8e5220393b40a4)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:49.477906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:49.482910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c822f2-1414-4ab6-a371-24f325e661d9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:49.505573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c9732-f1cf-4b15-9b0a-cd8a22584994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:49.506960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c9732-f1cf-4b15-9b0a-cd8a22584994 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:49.507449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c9732-f1cf-4b15-9b0a-cd8a22584994 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:49.508200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c9732-f1cf-4b15-9b0a-cd8a22584994 admin admin] PATH_INFO: `/v3/projects/09cf401afc3a45218b219083d0e31100/users/f07c9f83691449dd8686a68434d8f257/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:49.518450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c9732-f1cf-4b15-9b0a-cd8a22584994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cf401afc3a45218b219083d0e31100, user_id=f07c9f83691449dd8686a68434d8f257, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:49.526332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c9732-f1cf-4b15-9b0a-cd8a22584994 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:49.529688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:49.549800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1122/2224] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:49 2022] PUT /identity/v3/projects/09cf401afc3a45218b219083d0e31100/users/f07c9f83691449dd8686a68434d8f257/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:49.559427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb9ca2-d639-4e62-a7d0-21621f2b0e5d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:49.559721 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb9ca2-d639-4e62-a7d0-21621f2b0e5d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:49.560023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb9ca2-d639-4e62-a7d0-21621f2b0e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:49.598987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4a9f0a559b44b89f8e5220393b40a4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:49.610713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2adb9ca2-d639-4e62-a7d0-21621f2b0e5d None None] MFA Rules not processed for user `f07c9f83691449dd8686a68434d8f257`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:49.612180 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2adb9ca2-d639-4e62-a7d0-21621f2b0e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:49.614677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4a9f0a559b44b89f8e5220393b40a4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:49.616292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4a9f0a559b44b89f8e5220393b40a4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:49.640959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4a9f0a559b44b89f8e5220393b40a4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:49.646846 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:49.658238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4a9f0a559b44b89f8e5220393b40a4'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:49.668632 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1103/2225] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:49 2022] DELETE /identity/v3/users/1b4a9f0a559b44b89f8e5220393b40a4 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:49.712572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1123/2226] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:49.722736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:49.724280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:49.810655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f81f36-3c38-4a1e-9cb6-5a9a15b458a8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:49.816793 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be9139-dd7c-412b-8ec4-fa44a26bad91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:49.817600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be9139-dd7c-412b-8ec4-fa44a26bad91 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:49.817951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be9139-dd7c-412b-8ec4-fa44a26bad91 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:49.818462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be9139-dd7c-412b-8ec4-fa44a26bad91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:49.820553 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c1be9139-dd7c-412b-8ec4-fa44a26bad91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:49.911888 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be9139-dd7c-412b-8ec4-fa44a26bad91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:49.913393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be9139-dd7c-412b-8ec4-fa44a26bad91 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:49.923687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1104/2227] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:50.043769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb9ca2-d639-4e62-a7d0-21621f2b0e5d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:50.045276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2adb9ca2-d639-4e62-a7d0-21621f2b0e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:50.128059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb9ca2-d639-4e62-a7d0-21621f2b0e5d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:50.213515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:50.214435 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:50.217483 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:50.217822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0 admin admin] PATH_INFO: `/v3/projects/1b32087ea85a4f6d9bdc5a260c353229` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:50.219305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b32087ea85a4f6d9bdc5a260c353229)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:50.222004 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:50.281294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b32087ea85a4f6d9bdc5a260c353229', 'request_id': 'req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:50.292975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b32087ea85a4f6d9bdc5a260c353229', 'request_id': 'req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:50.311563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b32087ea85a4f6d9bdc5a260c353229', 'request_id': 'req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:50.328819 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:50.335239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0 admin admin] The token cache is being invalidate because project 1b32087ea85a4f6d9bdc5a260c353229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:50.338151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:50.351861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f25f6a16-f864-4fa2-9ed0-353e3662f3c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:50.351861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1124/2228] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:50 2022] DELETE /identity/v3/projects/1b32087ea85a4f6d9bdc5a260c353229 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:50.352473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8559fe-4cbc-485f-bade-16e8e20e14c3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:50.352473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8559fe-4cbc-485f-bade-16e8e20e14c3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:50.352473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8559fe-4cbc-485f-bade-16e8e20e14c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:50.374420 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-fc8559fe-4cbc-485f-bade-16e8e20e14c3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:50.375892 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fc8559fe-4cbc-485f-bade-16e8e20e14c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:50.438357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:50.456970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1105/2229] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:50.465015 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce076b28-4ba4-4fc8-8fb6-eac0337b7028 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:50.465498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce076b28-4ba4-4fc8-8fb6-eac0337b7028 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:50.465579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce076b28-4ba4-4fc8-8fb6-eac0337b7028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:50.483516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-ce076b28-4ba4-4fc8-8fb6-eac0337b7028 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:50.484756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ce076b28-4ba4-4fc8-8fb6-eac0337b7028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:50.492901 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1125/2230] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:50.498555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8559fe-4cbc-485f-bade-16e8e20e14c3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:50.512732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8559fe-4cbc-485f-bade-16e8e20e14c3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:50.519460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e69802-607f-45b7-b71e-d35dc8d4f83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:50.520557 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e69802-607f-45b7-b71e-d35dc8d4f83b admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:50.520557 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e69802-607f-45b7-b71e-d35dc8d4f83b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:50.520761 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e69802-607f-45b7-b71e-d35dc8d4f83b admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:50.521579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e69802-607f-45b7-b71e-d35dc8d4f83b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:50:50.521958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e69802-607f-45b7-b71e-d35dc8d4f83b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:50.523639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e69802-607f-45b7-b71e-d35dc8d4f83b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:50.537925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1106/2231] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:50:50.542755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce076b28-4ba4-4fc8-8fb6-eac0337b7028 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:50.552332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce076b28-4ba4-4fc8-8fb6-eac0337b7028 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:50.574804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fba42-fb65-464a-8231-5d6fa6a271df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:50.579733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fba42-fb65-464a-8231-5d6fa6a271df admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:50.580304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fba42-fb65-464a-8231-5d6fa6a271df admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:50.582466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fba42-fb65-464a-8231-5d6fa6a271df admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:50.584556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fba42-fb65-464a-8231-5d6fa6a271df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:50.591468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fba42-fb65-464a-8231-5d6fa6a271df admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:50.636373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1126/2232] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:50 2022] POST /identity/v3/projects => generated 399 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:50.646505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e69802-607f-45b7-b71e-d35dc8d4f83b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:50.660930 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e69802-607f-45b7-b71e-d35dc8d4f83b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:50.728974 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7559c6-2125-40ca-9f42-0210caf930bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:50.730920 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7559c6-2125-40ca-9f42-0210caf930bd admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:50.731407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7559c6-2125-40ca-9f42-0210caf930bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:50.732357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7559c6-2125-40ca-9f42-0210caf930bd admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:50.733027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7559c6-2125-40ca-9f42-0210caf930bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:50.735750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7559c6-2125-40ca-9f42-0210caf930bd admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:50.760031 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1107/2233] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:50 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:50:50.764083 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fba42-fb65-464a-8231-5d6fa6a271df admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:50.776201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fba42-fb65-464a-8231-5d6fa6a271df admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:50.785478 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9613cd2-0599-4a27-b531-8854eafee3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:50.786700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9613cd2-0599-4a27-b531-8854eafee3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:50.787447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9613cd2-0599-4a27-b531-8854eafee3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:50.787929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9613cd2-0599-4a27-b531-8854eafee3e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:50.789865 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9613cd2-0599-4a27-b531-8854eafee3e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:50.791667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9613cd2-0599-4a27-b531-8854eafee3e8 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:50.803162 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1127/2234] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:50:50.808164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7559c6-2125-40ca-9f42-0210caf930bd admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:50.821069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7559c6-2125-40ca-9f42-0210caf930bd admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:50.827229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7277072-054c-4477-8719-03b6a9faf019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:50.827932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7277072-054c-4477-8719-03b6a9faf019 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:50.828279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7277072-054c-4477-8719-03b6a9faf019 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:50.828556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7277072-054c-4477-8719-03b6a9faf019 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:50.829341 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7277072-054c-4477-8719-03b6a9faf019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:50.831110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7277072-054c-4477-8719-03b6a9faf019 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:50.844814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1108/2235] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:50:50.849301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9613cd2-0599-4a27-b531-8854eafee3e8 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:50.861905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9613cd2-0599-4a27-b531-8854eafee3e8 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:50.868465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688af7a-9cd9-4389-9a52-5c629a0fb490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:50.869659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688af7a-9cd9-4389-9a52-5c629a0fb490 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:50.870184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688af7a-9cd9-4389-9a52-5c629a0fb490 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:50.870569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688af7a-9cd9-4389-9a52-5c629a0fb490 admin admin] PATH_INFO: `/v3/projects/da4976cea0554c09a71805b798b2fa58/users/39ec1923c50744f08442452bf9e77a42/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:50.887705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688af7a-9cd9-4389-9a52-5c629a0fb490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da4976cea0554c09a71805b798b2fa58, user_id=39ec1923c50744f08442452bf9e77a42, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:50.890852 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688af7a-9cd9-4389-9a52-5c629a0fb490 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:50.908465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1128/2236] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:50 2022] PUT /identity/v3/projects/da4976cea0554c09a71805b798b2fa58/users/39ec1923c50744f08442452bf9e77a42/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:50:50.919752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e1e50-b82f-4cbd-a9d0-7ba711aaee28 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:50.920123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e1e50-b82f-4cbd-a9d0-7ba711aaee28 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:50.920360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e1e50-b82f-4cbd-a9d0-7ba711aaee28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:50.967517 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-cb6e1e50-b82f-4cbd-a9d0-7ba711aaee28 None None] MFA Rules not processed for user `39ec1923c50744f08442452bf9e77a42`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:50.968901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cb6e1e50-b82f-4cbd-a9d0-7ba711aaee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:51.067946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1109/2237] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:51.075944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688af7a-9cd9-4389-9a52-5c629a0fb490 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:51.078039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8688af7a-9cd9-4389-9a52-5c629a0fb490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:51.116534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e1e50-b82f-4cbd-a9d0-7ba711aaee28 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:51.122315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cb6e1e50-b82f-4cbd-a9d0-7ba711aaee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:51.164726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688af7a-9cd9-4389-9a52-5c629a0fb490 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:51.170587 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e389952-f996-43b4-bf86-ed0e887e1c49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:51.172069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e389952-f996-43b4-bf86-ed0e887e1c49 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:51.172665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e389952-f996-43b4-bf86-ed0e887e1c49 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:51.173126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e389952-f996-43b4-bf86-ed0e887e1c49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:51.175617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:51.192227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e389952-f996-43b4-bf86-ed0e887e1c49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:51.193974 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e389952-f996-43b4-bf86-ed0e887e1c49 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:51.203784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e1e50-b82f-4cbd-a9d0-7ba711aaee28 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:51.209409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5d50a-24e8-4420-b103-ee513979df82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:51.210512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1129/2238] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:50:51.216885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5d50a-24e8-4420-b103-ee513979df82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:51.217369 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5d50a-24e8-4420-b103-ee513979df82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:51.217824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5d50a-24e8-4420-b103-ee513979df82 admin admin] PATH_INFO: `/v3/users/70ea64f67f264b20a7c556c625a64f80` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:51.219568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5d50a-24e8-4420-b103-ee513979df82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70ea64f67f264b20a7c556c625a64f80)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:51.222132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5d50a-24e8-4420-b103-ee513979df82 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:51.297681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-77f5d50a-24e8-4420-b103-ee513979df82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70ea64f67f264b20a7c556c625a64f80'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:51.328338 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-77f5d50a-24e8-4420-b103-ee513979df82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70ea64f67f264b20a7c556c625a64f80'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:51.329582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-77f5d50a-24e8-4420-b103-ee513979df82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70ea64f67f264b20a7c556c625a64f80'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:51.361429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-77f5d50a-24e8-4420-b103-ee513979df82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70ea64f67f264b20a7c556c625a64f80'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:51.376253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-77f5d50a-24e8-4420-b103-ee513979df82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70ea64f67f264b20a7c556c625a64f80'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:51.393561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1110/2239] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:50:51 2022] DELETE /identity/v3/users/70ea64f67f264b20a7c556c625a64f80 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:51.601756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e389952-f996-43b4-bf86-ed0e887e1c49 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:51.630114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e389952-f996-43b4-bf86-ed0e887e1c49 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:51.717720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19fe51-0e13-4c41-b0d7-bde1ef134006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:51.718475 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19fe51-0e13-4c41-b0d7-bde1ef134006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:51.718759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19fe51-0e13-4c41-b0d7-bde1ef134006 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:51.719150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19fe51-0e13-4c41-b0d7-bde1ef134006 admin admin] PATH_INFO: `/v3/projects/38b640d5f6734a6b95b46a6f9dd695ec` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:51.720421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19fe51-0e13-4c41-b0d7-bde1ef134006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b640d5f6734a6b95b46a6f9dd695ec)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:51.722391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19fe51-0e13-4c41-b0d7-bde1ef134006 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:51.766365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:51.774361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5d19fe51-0e13-4c41-b0d7-bde1ef134006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b640d5f6734a6b95b46a6f9dd695ec', 'request_id': 'req-5d19fe51-0e13-4c41-b0d7-bde1ef134006'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:51.784078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5d19fe51-0e13-4c41-b0d7-bde1ef134006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b640d5f6734a6b95b46a6f9dd695ec', 'request_id': 'req-5d19fe51-0e13-4c41-b0d7-bde1ef134006'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:51.790941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5d19fe51-0e13-4c41-b0d7-bde1ef134006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b640d5f6734a6b95b46a6f9dd695ec', 'request_id': 'req-5d19fe51-0e13-4c41-b0d7-bde1ef134006'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:51.801333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5d19fe51-0e13-4c41-b0d7-bde1ef134006 admin admin] The token cache is being invalidate because project 38b640d5f6734a6b95b46a6f9dd695ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:50:51.801621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5d19fe51-0e13-4c41-b0d7-bde1ef134006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:51.802245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5d19fe51-0e13-4c41-b0d7-bde1ef134006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:50:51.803803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1130/2240] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:50:51 2022] DELETE /identity/v3/projects/38b640d5f6734a6b95b46a6f9dd695ec => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:51.808983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855f5a2-ac56-4a1a-8649-adc45d0a1b10 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:51.809440 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855f5a2-ac56-4a1a-8649-adc45d0a1b10 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:51.809545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855f5a2-ac56-4a1a-8649-adc45d0a1b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:51.826915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9855f5a2-ac56-4a1a-8649-adc45d0a1b10 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:51.828212 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9855f5a2-ac56-4a1a-8649-adc45d0a1b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:51.889619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1111/2241] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:51.898803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b36ecc-232c-40b2-a4ab-4660298d5349 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:51.899193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b36ecc-232c-40b2-a4ab-4660298d5349 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:51.899473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b36ecc-232c-40b2-a4ab-4660298d5349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:51.918979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-47b36ecc-232c-40b2-a4ab-4660298d5349 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:51.920214 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-47b36ecc-232c-40b2-a4ab-4660298d5349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:51.929189 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1131/2242] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:51.933568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855f5a2-ac56-4a1a-8649-adc45d0a1b10 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:51.944114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9855f5a2-ac56-4a1a-8649-adc45d0a1b10 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:51.950131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a426745-b59c-4eb0-881f-cdbaba46f6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:51.950789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a426745-b59c-4eb0-881f-cdbaba46f6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:51.951073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a426745-b59c-4eb0-881f-cdbaba46f6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:51.951346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a426745-b59c-4eb0-881f-cdbaba46f6e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:51.952120 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a426745-b59c-4eb0-881f-cdbaba46f6e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:50:51.952432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a426745-b59c-4eb0-881f-cdbaba46f6e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:51.954062 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a426745-b59c-4eb0-881f-cdbaba46f6e1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:51.966720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1112/2243] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:50:51.971211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b36ecc-232c-40b2-a4ab-4660298d5349 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:51.981090 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b36ecc-232c-40b2-a4ab-4660298d5349 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:51.985849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770c3f3-b09e-4dcf-ac8b-ef8bb4063be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:51.986530 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770c3f3-b09e-4dcf-ac8b-ef8bb4063be7 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:51.986814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770c3f3-b09e-4dcf-ac8b-ef8bb4063be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:51.987085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770c3f3-b09e-4dcf-ac8b-ef8bb4063be7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:51.987803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770c3f3-b09e-4dcf-ac8b-ef8bb4063be7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:51.990032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770c3f3-b09e-4dcf-ac8b-ef8bb4063be7 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:52.017136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1132/2244] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:50:51 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:50:52.022339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a426745-b59c-4eb0-881f-cdbaba46f6e1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:52.036542 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a426745-b59c-4eb0-881f-cdbaba46f6e1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:52.083942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:52.095545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1d563-5818-431b-9f20-2a3a44df2955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:52.096256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1d563-5818-431b-9f20-2a3a44df2955 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:52.096546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1d563-5818-431b-9f20-2a3a44df2955 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:52.096876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1d563-5818-431b-9f20-2a3a44df2955 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:52.098292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1d563-5818-431b-9f20-2a3a44df2955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:52.101552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1d563-5818-431b-9f20-2a3a44df2955 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:52.124580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1113/2245] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:50:52 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:50:52.130658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770c3f3-b09e-4dcf-ac8b-ef8bb4063be7 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:52.145703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770c3f3-b09e-4dcf-ac8b-ef8bb4063be7 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:52.151194 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626a754-8b02-4471-9489-7db15e5ee189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:52.151892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626a754-8b02-4471-9489-7db15e5ee189 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:52.152206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626a754-8b02-4471-9489-7db15e5ee189 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:52.152479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626a754-8b02-4471-9489-7db15e5ee189 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:52.153239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626a754-8b02-4471-9489-7db15e5ee189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:52.154882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626a754-8b02-4471-9489-7db15e5ee189 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:52.165706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1133/2246] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:52.169762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1d563-5818-431b-9f20-2a3a44df2955 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:52.179852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1d563-5818-431b-9f20-2a3a44df2955 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:52.184861 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33008713-f54a-4b9f-ac1c-49546f0541e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:52.185523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33008713-f54a-4b9f-ac1c-49546f0541e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:52.185809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33008713-f54a-4b9f-ac1c-49546f0541e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:52.186086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33008713-f54a-4b9f-ac1c-49546f0541e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:52.186824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33008713-f54a-4b9f-ac1c-49546f0541e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:52.188481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33008713-f54a-4b9f-ac1c-49546f0541e1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:52.198524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1114/2247] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:50:52.202570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626a754-8b02-4471-9489-7db15e5ee189 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:52.213027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626a754-8b02-4471-9489-7db15e5ee189 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:52.218045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893e63e-b2bf-4d5d-9609-0fefcf7a8e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:52.218945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893e63e-b2bf-4d5d-9609-0fefcf7a8e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:52.219233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893e63e-b2bf-4d5d-9609-0fefcf7a8e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:52.219508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893e63e-b2bf-4d5d-9609-0fefcf7a8e9e admin admin] PATH_INFO: `/v3/projects/2ceaecc3b7fc4fed9dce1e3ab1014ed2/users/b10d2c10313c4be2a8c11cff1eed8852/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:52.234769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893e63e-b2bf-4d5d-9609-0fefcf7a8e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ceaecc3b7fc4fed9dce1e3ab1014ed2, user_id=b10d2c10313c4be2a8c11cff1eed8852, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:52.237487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893e63e-b2bf-4d5d-9609-0fefcf7a8e9e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:52.253263 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1134/2248] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:50:52 2022] PUT /identity/v3/projects/2ceaecc3b7fc4fed9dce1e3ab1014ed2/users/b10d2c10313c4be2a8c11cff1eed8852/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:50:52.260164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e84fe-e458-4dde-a47a-d630963c48fb None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:52.260458 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e84fe-e458-4dde-a47a-d630963c48fb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:52.260738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e84fe-e458-4dde-a47a-d630963c48fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:52.279809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ad4e84fe-e458-4dde-a47a-d630963c48fb None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:52.281063 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ad4e84fe-e458-4dde-a47a-d630963c48fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:52.339398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1115/2249] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:52.350381 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893e63e-b2bf-4d5d-9609-0fefcf7a8e9e admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:52.352363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7893e63e-b2bf-4d5d-9609-0fefcf7a8e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:52.412297 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893e63e-b2bf-4d5d-9609-0fefcf7a8e9e admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:52.416184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5b85-2be4-4a3f-a5c9-f3b90a4c2745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:52.416833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5b85-2be4-4a3f-a5c9-f3b90a4c2745 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:52.417118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5b85-2be4-4a3f-a5c9-f3b90a4c2745 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:52.417394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e5b85-2be4-4a3f-a5c9-f3b90a4c2745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:52.426640 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e5b85-2be4-4a3f-a5c9-f3b90a4c2745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:52.428113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e5b85-2be4-4a3f-a5c9-f3b90a4c2745 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:52.438940 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1135/2250] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:50:56.607344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58956724-4af7-496c-b872-afd1265af862 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:56.607842 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58956724-4af7-496c-b872-afd1265af862 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:56.608042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58956724-4af7-496c-b872-afd1265af862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:56.667034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-58956724-4af7-496c-b872-afd1265af862 None None] MFA Rules not processed for user `b10d2c10313c4be2a8c11cff1eed8852`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:56.668282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-58956724-4af7-496c-b872-afd1265af862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:56.779230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1116/2251] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:56.796177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5b85-2be4-4a3f-a5c9-f3b90a4c2745 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:56.797620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-991e5b85-2be4-4a3f-a5c9-f3b90a4c2745 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:56.877028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e5b85-2be4-4a3f-a5c9-f3b90a4c2745 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:56.882369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cadc2-63a3-42b1-88c8-0d833f4ae88f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:56.883023 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cadc2-63a3-42b1-88c8-0d833f4ae88f service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:56.883308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cadc2-63a3-42b1-88c8-0d833f4ae88f service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:56.883579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cadc2-63a3-42b1-88c8-0d833f4ae88f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:56.888747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:50:56.893712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cadc2-63a3-42b1-88c8-0d833f4ae88f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:56.895582 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9cadc2-63a3-42b1-88c8-0d833f4ae88f service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:56.909465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1136/2252] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:50:58.415545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca86e0-20a3-4c6f-bd8c-5b97870f084e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:58.416157 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca86e0-20a3-4c6f-bd8c-5b97870f084e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:58.416157 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca86e0-20a3-4c6f-bd8c-5b97870f084e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:58.438856 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d8ca86e0-20a3-4c6f-bd8c-5b97870f084e None None] MFA Rules not processed for user `b10d2c10313c4be2a8c11cff1eed8852`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:50:58.440249 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d8ca86e0-20a3-4c6f-bd8c-5b97870f084e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:58.447545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1117/2253] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:50:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:50:59.053453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cadc2-63a3-42b1-88c8-0d833f4ae88f service nova] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:50:59.054990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1c9cadc2-63a3-42b1-88c8-0d833f4ae88f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:59.143720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9cadc2-63a3-42b1-88c8-0d833f4ae88f service nova] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:50:59.149489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76800a-33c0-432a-bb5a-daffa991be5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:50:59.150660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76800a-33c0-432a-bb5a-daffa991be5c service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:50:59.151157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76800a-33c0-432a-bb5a-daffa991be5c service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:50:59.151848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76800a-33c0-432a-bb5a-daffa991be5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:50:59.155054 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6d76800a-33c0-432a-bb5a-daffa991be5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:50:59.233305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76800a-33c0-432a-bb5a-daffa991be5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:50:59.234914 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76800a-33c0-432a-bb5a-daffa991be5c service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:50:59.248183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1137/2254] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:50:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:00.315543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca86e0-20a3-4c6f-bd8c-5b97870f084e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:00.326098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca86e0-20a3-4c6f-bd8c-5b97870f084e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:00.329994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80641019-5e17-4322-bcb7-8a9fbdc29b66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:00.330665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80641019-5e17-4322-bcb7-8a9fbdc29b66 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:00.330952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80641019-5e17-4322-bcb7-8a9fbdc29b66 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:00.331233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80641019-5e17-4322-bcb7-8a9fbdc29b66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:00.333132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-80641019-5e17-4322-bcb7-8a9fbdc29b66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:00.403655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:00.424620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80641019-5e17-4322-bcb7-8a9fbdc29b66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:00.426437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80641019-5e17-4322-bcb7-8a9fbdc29b66 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:00.452331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1118/2255] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:08.116804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76800a-33c0-432a-bb5a-daffa991be5c service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:08.118427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6d76800a-33c0-432a-bb5a-daffa991be5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:08.133967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76800a-33c0-432a-bb5a-daffa991be5c service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:08.139112 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:08.140124 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:08.140384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:08.140664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] PATH_INFO: `/v3/users/c07f8303c56741b9a70e82144a57b796` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:08.141907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c07f8303c56741b9a70e82144a57b796)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:08.144316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:08.195932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c07f8303c56741b9a70e82144a57b796'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:08.196714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c07f8303c56741b9a70e82144a57b796'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:08.206425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c07f8303c56741b9a70e82144a57b796'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:08.214579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c07f8303c56741b9a70e82144a57b796'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:08.229648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c07f8303c56741b9a70e82144a57b796'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:08.239930 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1138/2256] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:51:08 2022] DELETE /identity/v3/users/c07f8303c56741b9a70e82144a57b796 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:08.244784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707ca6c-05d9-4eb7-9800-dda5bcf97280 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:08.245715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707ca6c-05d9-4eb7-9800-dda5bcf97280 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:08.245980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1707ca6c-05d9-4eb7-9800-dda5bcf97280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:08.266239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1707ca6c-05d9-4eb7-9800-dda5bcf97280 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:08.268141 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1707ca6c-05d9-4eb7-9800-dda5bcf97280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:08.349806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1119/2257] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:08.359841 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:08.361328 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:08.389579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:08.436735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becf6db-3953-4791-b642-e200e0b5946d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:08.441654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee6c5b-086d-40b6-97d2-f97587a8d25d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:08.442319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee6c5b-086d-40b6-97d2-f97587a8d25d service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:08.442736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee6c5b-086d-40b6-97d2-f97587a8d25d service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:08.443011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee6c5b-086d-40b6-97d2-f97587a8d25d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:08.452842 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee6c5b-086d-40b6-97d2-f97587a8d25d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:08.454327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee6c5b-086d-40b6-97d2-f97587a8d25d service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:08.465250 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1139/2258] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:51:08.883234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707ca6c-05d9-4eb7-9800-dda5bcf97280 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:08.887202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1707ca6c-05d9-4eb7-9800-dda5bcf97280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:08.910946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1707ca6c-05d9-4eb7-9800-dda5bcf97280 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:08.925685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:08.927294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:08.927564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:08.927855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979 admin admin] PATH_INFO: `/v3/projects/e7afaabd6e4d4ea3aebc76e7d86576ab` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:08.930071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7afaabd6e4d4ea3aebc76e7d86576ab)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:08.933665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:09.060568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7afaabd6e4d4ea3aebc76e7d86576ab', 'request_id': 'req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:09.077569 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7afaabd6e4d4ea3aebc76e7d86576ab', 'request_id': 'req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:09.088370 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7afaabd6e4d4ea3aebc76e7d86576ab', 'request_id': 'req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:09.123017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979 admin admin] The token cache is being invalidate because project e7afaabd6e4d4ea3aebc76e7d86576ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:51:09.123315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:09.124055 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7bfbf229-3939-48c3-a7d9-bd2beb4f1979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:09.126116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1120/2259] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:51:08 2022] DELETE /identity/v3/projects/e7afaabd6e4d4ea3aebc76e7d86576ab => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:09.132184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c74a3d-14fe-4255-8ff1-b3dcfd4df5ab None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:09.132698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c74a3d-14fe-4255-8ff1-b3dcfd4df5ab None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:09.132924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c74a3d-14fe-4255-8ff1-b3dcfd4df5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:09.162488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-e0c74a3d-14fe-4255-8ff1-b3dcfd4df5ab None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:09.164460 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e0c74a3d-14fe-4255-8ff1-b3dcfd4df5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:09.239182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1140/2260] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:09.248274 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ff530-170f-47d8-b382-7d0329bf2d0c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:09.248844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ff530-170f-47d8-b382-7d0329bf2d0c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:09.248998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ff530-170f-47d8-b382-7d0329bf2d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:09.273259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-719ff530-170f-47d8-b382-7d0329bf2d0c None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:09.275087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-719ff530-170f-47d8-b382-7d0329bf2d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:09.289120 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1121/2261] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:09.294833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c74a3d-14fe-4255-8ff1-b3dcfd4df5ab None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:09.310086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c74a3d-14fe-4255-8ff1-b3dcfd4df5ab None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:09.337953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ccff6-8618-4d98-a069-e0a550ace0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:09.338814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ccff6-8618-4d98-a069-e0a550ace0bf admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:09.339400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ccff6-8618-4d98-a069-e0a550ace0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:09.341365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ccff6-8618-4d98-a069-e0a550ace0bf admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:09.344350 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ccff6-8618-4d98-a069-e0a550ace0bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:51:09.344687 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ccff6-8618-4d98-a069-e0a550ace0bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:09.348629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ccff6-8618-4d98-a069-e0a550ace0bf admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:09.384575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ff530-170f-47d8-b382-7d0329bf2d0c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:09.399107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1141/2262] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:51:09.410207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ff530-170f-47d8-b382-7d0329bf2d0c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:09.422530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba36a4b-fab8-4b6e-a339-5494cdad62c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:09.423275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba36a4b-fab8-4b6e-a339-5494cdad62c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:09.423945 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba36a4b-fab8-4b6e-a339-5494cdad62c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:09.424567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba36a4b-fab8-4b6e-a339-5494cdad62c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:09.425200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba36a4b-fab8-4b6e-a339-5494cdad62c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:09.427143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba36a4b-fab8-4b6e-a339-5494cdad62c7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:09.486294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1122/2263] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:51:09 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:51:09.492401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ccff6-8618-4d98-a069-e0a550ace0bf admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:09.509488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ccff6-8618-4d98-a069-e0a550ace0bf admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:09.591373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd892d8a-12d5-459c-87cd-6ab7d77bf145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:09.592535 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd892d8a-12d5-459c-87cd-6ab7d77bf145 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:09.592824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd892d8a-12d5-459c-87cd-6ab7d77bf145 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:09.593104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd892d8a-12d5-459c-87cd-6ab7d77bf145 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:09.593808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd892d8a-12d5-459c-87cd-6ab7d77bf145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:09.596072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd892d8a-12d5-459c-87cd-6ab7d77bf145 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:09.632287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1142/2264] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:51:09 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:51:09.634033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba36a4b-fab8-4b6e-a339-5494cdad62c7 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:09.659014 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba36a4b-fab8-4b6e-a339-5494cdad62c7 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:09.673146 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442a041-0a3c-4bf7-a013-be837c2f92b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:09.674000 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442a041-0a3c-4bf7-a013-be837c2f92b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:09.674354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442a041-0a3c-4bf7-a013-be837c2f92b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:09.674715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442a041-0a3c-4bf7-a013-be837c2f92b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:09.675488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442a041-0a3c-4bf7-a013-be837c2f92b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:09.677149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442a041-0a3c-4bf7-a013-be837c2f92b7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:09.693612 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1123/2265] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:51:09.694864 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd892d8a-12d5-459c-87cd-6ab7d77bf145 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:09.706423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd892d8a-12d5-459c-87cd-6ab7d77bf145 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:09.720099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6526ef-45cd-41a4-b3d9-ac2ea41c32a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:09.721285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6526ef-45cd-41a4-b3d9-ac2ea41c32a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:09.721677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6526ef-45cd-41a4-b3d9-ac2ea41c32a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:09.722073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6526ef-45cd-41a4-b3d9-ac2ea41c32a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:09.722964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6526ef-45cd-41a4-b3d9-ac2ea41c32a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:09.724895 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6526ef-45cd-41a4-b3d9-ac2ea41c32a3 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:09.738601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1143/2266] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:51:09.742940 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442a041-0a3c-4bf7-a013-be837c2f92b7 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:09.753941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442a041-0a3c-4bf7-a013-be837c2f92b7 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:09.759017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3f658-73d0-4767-a70f-4e9ec5391e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:09.760191 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3f658-73d0-4767-a70f-4e9ec5391e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:09.760420 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3f658-73d0-4767-a70f-4e9ec5391e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:09.760757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3f658-73d0-4767-a70f-4e9ec5391e3f admin admin] PATH_INFO: `/v3/projects/3e33bfd157c64f8688b9720d80a3a8af/users/2ea01587de8f4d5584d1b12ef042d421/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:09.777038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3f658-73d0-4767-a70f-4e9ec5391e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e33bfd157c64f8688b9720d80a3a8af, user_id=2ea01587de8f4d5584d1b12ef042d421, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:09.780645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3f658-73d0-4767-a70f-4e9ec5391e3f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:09.806119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1124/2267] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:51:09 2022] PUT /identity/v3/projects/3e33bfd157c64f8688b9720d80a3a8af/users/2ea01587de8f4d5584d1b12ef042d421/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:09.808037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5a1e8-e551-41e9-a7f5-0ab10384a696 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:09.808674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5a1e8-e551-41e9-a7f5-0ab10384a696 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:09.812168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5a1e8-e551-41e9-a7f5-0ab10384a696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:09.855351 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-72e5a1e8-e551-41e9-a7f5-0ab10384a696 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:09.856710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-72e5a1e8-e551-41e9-a7f5-0ab10384a696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:09.921680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:09.982355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1144/2268] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:09.993808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3f658-73d0-4767-a70f-4e9ec5391e3f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:09.996470 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e8b3f658-73d0-4767-a70f-4e9ec5391e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:10.019288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:10.101744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5a1e8-e551-41e9-a7f5-0ab10384a696 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:10.103209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-72e5a1e8-e551-41e9-a7f5-0ab10384a696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:10.104598 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3f658-73d0-4767-a70f-4e9ec5391e3f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:10.108556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0846ae-8a7d-47be-bf73-ec9c1e6d19c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:10.109229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0846ae-8a7d-47be-bf73-ec9c1e6d19c6 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:10.109682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0846ae-8a7d-47be-bf73-ec9c1e6d19c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:10.109966 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0846ae-8a7d-47be-bf73-ec9c1e6d19c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:10.116426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5a1e8-e551-41e9-a7f5-0ab10384a696 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:10.121916 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:10.122867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:10.123215 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:10.123502 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0846ae-8a7d-47be-bf73-ec9c1e6d19c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:10.123626 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] PATH_INFO: `/v3/users/39ec1923c50744f08442452bf9e77a42` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:10.125042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39ec1923c50744f08442452bf9e77a42)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:10.127227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0846ae-8a7d-47be-bf73-ec9c1e6d19c6 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:10.127627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:10.149896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1125/2269] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:10.224199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39ec1923c50744f08442452bf9e77a42'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:10.225018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39ec1923c50744f08442452bf9e77a42'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:10.235060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39ec1923c50744f08442452bf9e77a42'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:10.249543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39ec1923c50744f08442452bf9e77a42'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:10.277493 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39ec1923c50744f08442452bf9e77a42'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:10.284642 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1145/2270] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:51:10 2022] DELETE /identity/v3/users/39ec1923c50744f08442452bf9e77a42 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:51:10.289379 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb7dfd-ec44-47bb-9f37-e0a5a2b424a3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:10.289880 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb7dfd-ec44-47bb-9f37-e0a5a2b424a3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:10.290287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb7dfd-ec44-47bb-9f37-e0a5a2b424a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:10.314526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4cfb7dfd-ec44-47bb-9f37-e0a5a2b424a3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:10.316447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4cfb7dfd-ec44-47bb-9f37-e0a5a2b424a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:10.392225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1126/2271] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:51:10.408507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:10.409977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:10.484996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f66ce-96be-49e7-b262-d40c9a64acd3 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:10.489385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ea84f-ce3b-4a96-ad67-0944460834fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:10.490047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ea84f-ce3b-4a96-ad67-0944460834fb service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:10.490505 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ea84f-ce3b-4a96-ad67-0944460834fb service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:10.490781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ea84f-ce3b-4a96-ad67-0944460834fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:10.507060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ea84f-ce3b-4a96-ad67-0944460834fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:10.508561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ea84f-ce3b-4a96-ad67-0944460834fb service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:10.545448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1146/2272] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:10.946226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb7dfd-ec44-47bb-9f37-e0a5a2b424a3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:10.947987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4cfb7dfd-ec44-47bb-9f37-e0a5a2b424a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:10.966752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb7dfd-ec44-47bb-9f37-e0a5a2b424a3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:10.974400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:10.975486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:10.980103 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:10.980664 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] PATH_INFO: `/v3/projects/da4976cea0554c09a71805b798b2fa58` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:10.983483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=da4976cea0554c09a71805b798b2fa58)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:10.985309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:11.028064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58ff14-1746-4c37-a7d1-de35412743d4 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:11.028374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58ff14-1746-4c37-a7d1-de35412743d4 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:11.028621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58ff14-1746-4c37-a7d1-de35412743d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:11.057955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-9b58ff14-1746-4c37-a7d1-de35412743d4 None None] MFA Rules not processed for user `2ea01587de8f4d5584d1b12ef042d421`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:11.058236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da4976cea0554c09a71805b798b2fa58', 'request_id': 'req-1a38036b-cfdb-43e7-897e-db9c131d7e4d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:11.059265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-9b58ff14-1746-4c37-a7d1-de35412743d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:11.071319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da4976cea0554c09a71805b798b2fa58', 'request_id': 'req-1a38036b-cfdb-43e7-897e-db9c131d7e4d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:11.078449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da4976cea0554c09a71805b798b2fa58', 'request_id': 'req-1a38036b-cfdb-43e7-897e-db9c131d7e4d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:11.096116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] The token cache is being invalidate because project da4976cea0554c09a71805b798b2fa58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:51:11.096898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:11.097378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:11.099289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:11.116037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1127/2273] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:51:10 2022] DELETE /identity/v3/projects/da4976cea0554c09a71805b798b2fa58 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:11.142157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1147/2274] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:51:11.154153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:11.155630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:11.200879 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:11.230531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38036b-cfdb-43e7-897e-db9c131d7e4d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:11.234719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1c117-5198-417f-989e-7844eee17f25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:11.235472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1c117-5198-417f-989e-7844eee17f25 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:11.235761 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1c117-5198-417f-989e-7844eee17f25 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:11.236066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1c117-5198-417f-989e-7844eee17f25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:11.246123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1c117-5198-417f-989e-7844eee17f25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:11.247615 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1c117-5198-417f-989e-7844eee17f25 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:11.258213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1128/2275] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:12.368806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58ff14-1746-4c37-a7d1-de35412743d4 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:12.370382 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-9b58ff14-1746-4c37-a7d1-de35412743d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:12.441988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58ff14-1746-4c37-a7d1-de35412743d4 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:12.446940 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:12.447814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:12.448135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:12.448395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] PATH_INFO: `/v3/users/2be338cf0ed24d8095b71980e42e5fac` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:12.449605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2be338cf0ed24d8095b71980e42e5fac)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:12.451706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:12.501988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2be338cf0ed24d8095b71980e42e5fac'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:12.502630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2be338cf0ed24d8095b71980e42e5fac'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:12.511228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2be338cf0ed24d8095b71980e42e5fac'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:12.519171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2be338cf0ed24d8095b71980e42e5fac'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:12.531729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2be338cf0ed24d8095b71980e42e5fac'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:12.542672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1148/2276] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:51:12 2022] DELETE /identity/v3/users/2be338cf0ed24d8095b71980e42e5fac => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:12.547567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1b22-4137-4e90-9140-c664620d9a53 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:12.548178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1b22-4137-4e90-9140-c664620d9a53 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:12.548279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1b22-4137-4e90-9140-c664620d9a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:12.567188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-babc1b22-4137-4e90-9140-c664620d9a53 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:12.568775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-babc1b22-4137-4e90-9140-c664620d9a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:12.647865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1129/2277] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:12.658030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:12.659739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:12.721646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd0700-68aa-48f5-bfe1-2582fe707b30 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:12.725348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3ffdf-4f8d-40ce-bc20-e8a0919d12b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:12.726011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3ffdf-4f8d-40ce-bc20-e8a0919d12b5 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:12.726308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3ffdf-4f8d-40ce-bc20-e8a0919d12b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:12.726571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3ffdf-4f8d-40ce-bc20-e8a0919d12b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:12.733854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:12.736363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3ffdf-4f8d-40ce-bc20-e8a0919d12b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:12.738551 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3ffdf-4f8d-40ce-bc20-e8a0919d12b5 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:12.750739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1149/2278] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:13.025422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1b22-4137-4e90-9140-c664620d9a53 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:13.026960 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-babc1b22-4137-4e90-9140-c664620d9a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:13.040810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1b22-4137-4e90-9140-c664620d9a53 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:13.046480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:13.047493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.047782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.048086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] PATH_INFO: `/v3/projects/64b5c22a5a7447718164af38b437c405` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.049304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] RBAC: Authorizing `identity:delete_project(project_id=64b5c22a5a7447718164af38b437c405)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:13.051248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:13.100013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64b5c22a5a7447718164af38b437c405', 'request_id': 'req-3cf9141c-eba0-489e-a99c-9eb3b214e87c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:13.108081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64b5c22a5a7447718164af38b437c405', 'request_id': 'req-3cf9141c-eba0-489e-a99c-9eb3b214e87c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:13.115379 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64b5c22a5a7447718164af38b437c405', 'request_id': 'req-3cf9141c-eba0-489e-a99c-9eb3b214e87c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:13.127533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] The token cache is being invalidate because project 64b5c22a5a7447718164af38b437c405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:51:13.128096 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:13.128756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:13.130336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1130/2279] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:51:13 2022] DELETE /identity/v3/projects/64b5c22a5a7447718164af38b437c405 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:13.138736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679391b5-0a59-4950-9211-dadcfa1dcff0 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.139026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679391b5-0a59-4950-9211-dadcfa1dcff0 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.139305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679391b5-0a59-4950-9211-dadcfa1dcff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.160225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-679391b5-0a59-4950-9211-dadcfa1dcff0 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:13.161985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-679391b5-0a59-4950-9211-dadcfa1dcff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:13.222878 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:13.224730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:13.314473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1150/2280] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:13.327537 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde3052-295b-4b00-9b25-b6b17e96c455 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.328248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde3052-295b-4b00-9b25-b6b17e96c455 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.328710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde3052-295b-4b00-9b25-b6b17e96c455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.339365 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9141c-eba0-489e-a99c-9eb3b214e87c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:13.344619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbbc9f-26e9-4279-b090-cbdae8afe824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:13.345631 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbbc9f-26e9-4279-b090-cbdae8afe824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.345988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbbc9f-26e9-4279-b090-cbdae8afe824 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.346337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbbc9f-26e9-4279-b090-cbdae8afe824 admin admin] PATH_INFO: `/v3/users/d3ec9afe54cf479888a6af7eb3a7dd72` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.347682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbbc9f-26e9-4279-b090-cbdae8afe824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ec9afe54cf479888a6af7eb3a7dd72)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:13.350483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbbc9f-26e9-4279-b090-cbdae8afe824 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:13.352304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:13.369893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-1bde3052-295b-4b00-9b25-b6b17e96c455 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:13.371793 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1bde3052-295b-4b00-9b25-b6b17e96c455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:13.393951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1151/2281] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:51:13.399303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde3052-295b-4b00-9b25-b6b17e96c455 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:13.416952 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde3052-295b-4b00-9b25-b6b17e96c455 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:13.427847 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c8cb1-f57f-4e13-96f2-41e3350027a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:13.429362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c8cb1-f57f-4e13-96f2-41e3350027a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.429362 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c8cb1-f57f-4e13-96f2-41e3350027a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.429772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c8cb1-f57f-4e13-96f2-41e3350027a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.431218 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c8cb1-f57f-4e13-96f2-41e3350027a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:51:13.431672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c8cb1-f57f-4e13-96f2-41e3350027a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:13.444607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c8cb1-f57f-4e13-96f2-41e3350027a9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:13.460157 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-95cbbc9f-26e9-4279-b090-cbdae8afe824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ec9afe54cf479888a6af7eb3a7dd72'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:13.472522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1152/2282] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:51:13.476742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c8cb1-f57f-4e13-96f2-41e3350027a9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:13.481235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-95cbbc9f-26e9-4279-b090-cbdae8afe824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ec9afe54cf479888a6af7eb3a7dd72'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:13.482508 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-95cbbc9f-26e9-4279-b090-cbdae8afe824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ec9afe54cf479888a6af7eb3a7dd72'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:13.489621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5d9c8cb1-f57f-4e13-96f2-41e3350027a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:13.493931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-95cbbc9f-26e9-4279-b090-cbdae8afe824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ec9afe54cf479888a6af7eb3a7dd72'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:13.521815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c8cb1-f57f-4e13-96f2-41e3350027a9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:13.526080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-95cbbc9f-26e9-4279-b090-cbdae8afe824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ec9afe54cf479888a6af7eb3a7dd72'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:13.537183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87543663-2e01-4078-a9cd-70ddbc7b14d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:13.538111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87543663-2e01-4078-a9cd-70ddbc7b14d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.538282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1131/2283] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:51:13 2022] DELETE /identity/v3/users/d3ec9afe54cf479888a6af7eb3a7dd72 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:13.538435 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87543663-2e01-4078-a9cd-70ddbc7b14d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.538772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87543663-2e01-4078-a9cd-70ddbc7b14d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.540124 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87543663-2e01-4078-a9cd-70ddbc7b14d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:13.542202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4523271-fb2a-4db7-9f20-a428f54d3924 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.542497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4523271-fb2a-4db7-9f20-a428f54d3924 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.544259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4523271-fb2a-4db7-9f20-a428f54d3924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.546998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87543663-2e01-4078-a9cd-70ddbc7b14d1 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:13.565476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e4523271-fb2a-4db7-9f20-a428f54d3924 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:13.568641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e4523271-fb2a-4db7-9f20-a428f54d3924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:13.607376 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1153/2284] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:51:13 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:51:13.612394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87543663-2e01-4078-a9cd-70ddbc7b14d1 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:13.694381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1132/2285] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:13.711841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43b82c-4be5-4253-a011-4cde3e22f74c None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.712198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43b82c-4be5-4253-a011-4cde3e22f74c None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.712482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43b82c-4be5-4253-a011-4cde3e22f74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.741849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-af43b82c-4be5-4253-a011-4cde3e22f74c None None] MFA Rules not processed for user `2ea01587de8f4d5584d1b12ef042d421`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:13.743108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-af43b82c-4be5-4253-a011-4cde3e22f74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:13.745154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87543663-2e01-4078-a9cd-70ddbc7b14d1 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:13.760889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b769d-be7e-4753-ba43-6d37dff25c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:13.762272 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b769d-be7e-4753-ba43-6d37dff25c6f admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.762588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b769d-be7e-4753-ba43-6d37dff25c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.762837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b769d-be7e-4753-ba43-6d37dff25c6f admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.763693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b769d-be7e-4753-ba43-6d37dff25c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:13.766713 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b769d-be7e-4753-ba43-6d37dff25c6f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:13.823750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1154/2286] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:51:13 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:51:13.825535 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b769d-be7e-4753-ba43-6d37dff25c6f admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:13.828591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7f8b769d-be7e-4753-ba43-6d37dff25c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:13.851773 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1133/2287] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:13.851773 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43b82c-4be5-4253-a011-4cde3e22f74c None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:13.864024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43b82c-4be5-4253-a011-4cde3e22f74c None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:13.870173 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f81f9-b20a-4d44-a852-c151e06c7d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:13.871098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f81f9-b20a-4d44-a852-c151e06c7d69 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.871372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f81f9-b20a-4d44-a852-c151e06c7d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.871749 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f81f9-b20a-4d44-a852-c151e06c7d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.873159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f81f9-b20a-4d44-a852-c151e06c7d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:13.876477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f81f9-b20a-4d44-a852-c151e06c7d69 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:13.891066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1134/2288] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:51:13.895674 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f81f9-b20a-4d44-a852-c151e06c7d69 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:13.915756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f81f9-b20a-4d44-a852-c151e06c7d69 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:13.922958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38df7e6-9573-4f22-8c63-2cae0029398d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:13.923794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38df7e6-9573-4f22-8c63-2cae0029398d admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.924205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38df7e6-9573-4f22-8c63-2cae0029398d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.924577 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38df7e6-9573-4f22-8c63-2cae0029398d admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.925412 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38df7e6-9573-4f22-8c63-2cae0029398d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:13.927160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38df7e6-9573-4f22-8c63-2cae0029398d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:13.952615 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1135/2289] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:51:13.952822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:13.954022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38df7e6-9573-4f22-8c63-2cae0029398d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:13.967954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:13.972810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b769d-be7e-4753-ba43-6d37dff25c6f admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:13.974290 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38df7e6-9573-4f22-8c63-2cae0029398d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:13.980844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb719b2-bdbf-4df6-ade7-48e10b5d16cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:13.981070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc08e5-8ec5-43ea-bb98-263b3331692f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:13.982214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc08e5-8ec5-43ea-bb98-263b3331692f admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.982830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb719b2-bdbf-4df6-ade7-48e10b5d16cf service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:13.983366 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc08e5-8ec5-43ea-bb98-263b3331692f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.983485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb719b2-bdbf-4df6-ade7-48e10b5d16cf service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:13.983601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc08e5-8ec5-43ea-bb98-263b3331692f admin admin] PATH_INFO: `/v3/projects/487d80530bdc47d68d3dce4a978ca6e9/users/99d82e4f5e90442fae210efc2ba5e44f/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.983738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb719b2-bdbf-4df6-ade7-48e10b5d16cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:13.996111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb719b2-bdbf-4df6-ade7-48e10b5d16cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:13.998219 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb719b2-bdbf-4df6-ade7-48e10b5d16cf service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:14.011497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc08e5-8ec5-43ea-bb98-263b3331692f admin admin] RBAC: Authorizing `identity:create_grant(project_id=487d80530bdc47d68d3dce4a978ca6e9, user_id=99d82e4f5e90442fae210efc2ba5e44f, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:14.015374 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1155/2290] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:14.016272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc08e5-8ec5-43ea-bb98-263b3331692f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:14.040862 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1136/2291] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:51:13 2022] PUT /identity/v3/projects/487d80530bdc47d68d3dce4a978ca6e9/users/99d82e4f5e90442fae210efc2ba5e44f/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:14.045677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc690d-9950-4502-9eed-b790a40dbaa9 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:14.046694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc690d-9950-4502-9eed-b790a40dbaa9 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:14.046694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc690d-9950-4502-9eed-b790a40dbaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:14.077756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a4fc690d-9950-4502-9eed-b790a40dbaa9 None None] MFA Rules not processed for user `99d82e4f5e90442fae210efc2ba5e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:14.079146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a4fc690d-9950-4502-9eed-b790a40dbaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:14.151873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1156/2292] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:14.163917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc08e5-8ec5-43ea-bb98-263b3331692f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:14.165707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d1fc08e5-8ec5-43ea-bb98-263b3331692f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:14.231796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc08e5-8ec5-43ea-bb98-263b3331692f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:14.235671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35eef0-7b0e-4511-9821-89eb573b5ad6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7297224dc73c4358b60f1b096c0fb5bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:14.236461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35eef0-7b0e-4511-9821-89eb573b5ad6 service glance] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:14.236812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35eef0-7b0e-4511-9821-89eb573b5ad6 service glance] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:14.237161 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35eef0-7b0e-4511-9821-89eb573b5ad6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:14.247035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35eef0-7b0e-4511-9821-89eb573b5ad6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:14.248914 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35eef0-7b0e-4511-9821-89eb573b5ad6 service glance] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:14.259822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1137/2293] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:14.993662 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc690d-9950-4502-9eed-b790a40dbaa9 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:14.995528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a4fc690d-9950-4502-9eed-b790a40dbaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:15.091992 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc690d-9950-4502-9eed-b790a40dbaa9 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:15.097321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:15.098202 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:15.098482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:15.098878 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] PATH_INFO: `/v3/projects/fe0e4ddd0d164c66962a1de9e9261c3e` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:15.100130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe0e4ddd0d164c66962a1de9e9261c3e)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:15.102057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:15.159501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe0e4ddd0d164c66962a1de9e9261c3e', 'request_id': 'req-5671c89d-f0de-4250-894a-9c370b4d35c7'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:15.170184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe0e4ddd0d164c66962a1de9e9261c3e', 'request_id': 'req-5671c89d-f0de-4250-894a-9c370b4d35c7'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:15.180421 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe0e4ddd0d164c66962a1de9e9261c3e', 'request_id': 'req-5671c89d-f0de-4250-894a-9c370b4d35c7'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:15.212295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] The token cache is being invalidate because project fe0e4ddd0d164c66962a1de9e9261c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:51:15.212470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:15.213412 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:15.215069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1157/2294] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:51:14 2022] DELETE /identity/v3/projects/fe0e4ddd0d164c66962a1de9e9261c3e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:15.219709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35eef0-7b0e-4511-9821-89eb573b5ad6 service glance] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:15.231248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35eef0-7b0e-4511-9821-89eb573b5ad6 service glance] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:15.299977 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:15.300406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:15.300647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:15.300929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] PATH_INFO: `/v3/users/c3fa169998114da19e4dcc8dec432150` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:15.302248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3fa169998114da19e4dcc8dec432150)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:15.304342 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:15.360849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3fa169998114da19e4dcc8dec432150'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:15.369818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3fa169998114da19e4dcc8dec432150'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:15.370648 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3fa169998114da19e4dcc8dec432150'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:15.383266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3fa169998114da19e4dcc8dec432150'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:15.398130 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3fa169998114da19e4dcc8dec432150'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:15.400829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:15.406119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1138/2295] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:51:15 2022] DELETE /identity/v3/users/c3fa169998114da19e4dcc8dec432150 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:15.765090 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:15.766519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:15.793873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671c89d-f0de-4250-894a-9c370b4d35c7 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:15.813703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:15.915812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:15.917069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:15.917398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:15.917687 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] PATH_INFO: `/v3/projects/cb218c3cf4ba48b1bf5e154f525f1a35` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:15.918990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb218c3cf4ba48b1bf5e154f525f1a35)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:15.921112 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:16.001227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb218c3cf4ba48b1bf5e154f525f1a35', 'request_id': 'req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.010594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb218c3cf4ba48b1bf5e154f525f1a35', 'request_id': 'req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.017785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb218c3cf4ba48b1bf5e154f525f1a35', 'request_id': 'req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.028169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] The token cache is being invalidate because project cb218c3cf4ba48b1bf5e154f525f1a35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:51:16.028491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.029154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.030701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1158/2296] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:51:15 2022] DELETE /identity/v3/projects/cb218c3cf4ba48b1bf5e154f525f1a35 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:16.034791 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:16.036590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:16.053347 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2120c-fec4-4683-9d88-123342d52547 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:16.117765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93ea37-b130-4967-b27f-9d956a357dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:16.117765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93ea37-b130-4967-b27f-9d956a357dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:16.117765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93ea37-b130-4967-b27f-9d956a357dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:16.117765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93ea37-b130-4967-b27f-9d956a357dd0 admin admin] PATH_INFO: `/v3/users/f07c9f83691449dd8686a68434d8f257` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:16.118447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93ea37-b130-4967-b27f-9d956a357dd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f07c9f83691449dd8686a68434d8f257)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:16.120480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93ea37-b130-4967-b27f-9d956a357dd0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:16.168178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7a93ea37-b130-4967-b27f-9d956a357dd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f07c9f83691449dd8686a68434d8f257'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.177460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7a93ea37-b130-4967-b27f-9d956a357dd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f07c9f83691449dd8686a68434d8f257'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.178326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7a93ea37-b130-4967-b27f-9d956a357dd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f07c9f83691449dd8686a68434d8f257'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.188213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7a93ea37-b130-4967-b27f-9d956a357dd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f07c9f83691449dd8686a68434d8f257'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.204538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7a93ea37-b130-4967-b27f-9d956a357dd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f07c9f83691449dd8686a68434d8f257'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.211576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1139/2297] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:51:16 2022] DELETE /identity/v3/users/f07c9f83691449dd8686a68434d8f257 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:16.555945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:16.557759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:16.572046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1724a-6b80-4a1a-a377-a2dea5fcbb6f admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:16.631086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fc2d2-540e-471c-9a4c-2b46a963e3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:16.632445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fc2d2-540e-471c-9a4c-2b46a963e3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:16.632845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fc2d2-540e-471c-9a4c-2b46a963e3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:16.633201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fc2d2-540e-471c-9a4c-2b46a963e3cd admin admin] PATH_INFO: `/v3/projects/09cf401afc3a45218b219083d0e31100` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:16.635002 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fc2d2-540e-471c-9a4c-2b46a963e3cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=09cf401afc3a45218b219083d0e31100)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:16.637826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fc2d2-540e-471c-9a4c-2b46a963e3cd admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:16.690142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:16.702657 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-181fc2d2-540e-471c-9a4c-2b46a963e3cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09cf401afc3a45218b219083d0e31100', 'request_id': 'req-181fc2d2-540e-471c-9a4c-2b46a963e3cd'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.713103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-181fc2d2-540e-471c-9a4c-2b46a963e3cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09cf401afc3a45218b219083d0e31100', 'request_id': 'req-181fc2d2-540e-471c-9a4c-2b46a963e3cd'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.720077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-181fc2d2-540e-471c-9a4c-2b46a963e3cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09cf401afc3a45218b219083d0e31100', 'request_id': 'req-181fc2d2-540e-471c-9a4c-2b46a963e3cd'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.730417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-181fc2d2-540e-471c-9a4c-2b46a963e3cd admin admin] The token cache is being invalidate because project 09cf401afc3a45218b219083d0e31100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:51:16.730704 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-181fc2d2-540e-471c-9a4c-2b46a963e3cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.731347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-181fc2d2-540e-471c-9a4c-2b46a963e3cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:16.732947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1159/2298] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:51:16 2022] DELETE /identity/v3/projects/09cf401afc3a45218b219083d0e31100 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:16.738210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9839dff-e629-419f-b3fa-2a8b413e41e5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:16.738728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9839dff-e629-419f-b3fa-2a8b413e41e5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:16.738728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9839dff-e629-419f-b3fa-2a8b413e41e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:16.757740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c9839dff-e629-419f-b3fa-2a8b413e41e5 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:16.758948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c9839dff-e629-419f-b3fa-2a8b413e41e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:16.822544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1140/2299] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:16.830641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6775784-076b-4b55-8b4e-289801da7e45 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:16.830932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6775784-076b-4b55-8b4e-289801da7e45 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:16.831209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6775784-076b-4b55-8b4e-289801da7e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:16.850657 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a6775784-076b-4b55-8b4e-289801da7e45 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:16.851854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a6775784-076b-4b55-8b4e-289801da7e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:16.860021 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1160/2300] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:16.865448 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9839dff-e629-419f-b3fa-2a8b413e41e5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:16.875665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9839dff-e629-419f-b3fa-2a8b413e41e5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:16.880867 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85bcd2-0050-4f30-8c38-0824c3a8de3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:16.881526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85bcd2-0050-4f30-8c38-0824c3a8de3d admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:16.881826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85bcd2-0050-4f30-8c38-0824c3a8de3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:16.882103 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85bcd2-0050-4f30-8c38-0824c3a8de3d admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:16.882847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85bcd2-0050-4f30-8c38-0824c3a8de3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:51:16.883148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85bcd2-0050-4f30-8c38-0824c3a8de3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:16.884775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85bcd2-0050-4f30-8c38-0824c3a8de3d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:16.900686 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1141/2301] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:51:16.905233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6775784-076b-4b55-8b4e-289801da7e45 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:16.914602 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6775784-076b-4b55-8b4e-289801da7e45 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:16.920125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412b9b6-dab8-4b55-a3d0-6bcdee884669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:16.920938 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412b9b6-dab8-4b55-a3d0-6bcdee884669 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:16.921512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412b9b6-dab8-4b55-a3d0-6bcdee884669 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:16.921709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412b9b6-dab8-4b55-a3d0-6bcdee884669 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:16.922465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412b9b6-dab8-4b55-a3d0-6bcdee884669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:16.924495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412b9b6-dab8-4b55-a3d0-6bcdee884669 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:16.954562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1161/2302] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:51:16 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:51:16.958736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85bcd2-0050-4f30-8c38-0824c3a8de3d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:16.970981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85bcd2-0050-4f30-8c38-0824c3a8de3d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:16.975226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:17.035523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723f776-6ad4-465d-b8b3-cadb72ff4def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:17.036266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723f776-6ad4-465d-b8b3-cadb72ff4def admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:17.036694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723f776-6ad4-465d-b8b3-cadb72ff4def admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:17.036963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723f776-6ad4-465d-b8b3-cadb72ff4def admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:17.037657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723f776-6ad4-465d-b8b3-cadb72ff4def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:17.039596 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723f776-6ad4-465d-b8b3-cadb72ff4def admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:17.065457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1142/2303] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:51:16 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:51:17.067638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412b9b6-dab8-4b55-a3d0-6bcdee884669 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:17.078246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412b9b6-dab8-4b55-a3d0-6bcdee884669 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:17.083314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354f42d-05bb-44e3-b1af-16153ebf5750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:17.084225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354f42d-05bb-44e3-b1af-16153ebf5750 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:17.084682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354f42d-05bb-44e3-b1af-16153ebf5750 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:17.085024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354f42d-05bb-44e3-b1af-16153ebf5750 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:17.085767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354f42d-05bb-44e3-b1af-16153ebf5750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:17.087438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354f42d-05bb-44e3-b1af-16153ebf5750 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:17.098528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1162/2304] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:51:17.102926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723f776-6ad4-465d-b8b3-cadb72ff4def admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:17.113021 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723f776-6ad4-465d-b8b3-cadb72ff4def admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:17.118629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3275591-c8f1-4f33-9229-ee4b58801130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:17.119420 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3275591-c8f1-4f33-9229-ee4b58801130 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:17.119852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3275591-c8f1-4f33-9229-ee4b58801130 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:17.120234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3275591-c8f1-4f33-9229-ee4b58801130 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:17.121114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3275591-c8f1-4f33-9229-ee4b58801130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:17.122814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3275591-c8f1-4f33-9229-ee4b58801130 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:17.144127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1143/2305] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:51:17.144378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354f42d-05bb-44e3-b1af-16153ebf5750 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:17.154900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354f42d-05bb-44e3-b1af-16153ebf5750 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:17.163783 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dfaea-64fb-4755-9af0-32ba69fbf1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:17.164732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dfaea-64fb-4755-9af0-32ba69fbf1e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:17.165017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dfaea-64fb-4755-9af0-32ba69fbf1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:17.165295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dfaea-64fb-4755-9af0-32ba69fbf1e6 admin admin] PATH_INFO: `/v3/projects/581bcdfdd6e942d6a9a60d5f9ed5ef19/users/2780258f73ee4f5d8aeff7295d8f271c/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:17.182599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867dfaea-64fb-4755-9af0-32ba69fbf1e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=581bcdfdd6e942d6a9a60d5f9ed5ef19, user_id=2780258f73ee4f5d8aeff7295d8f271c, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:17.185414 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867dfaea-64fb-4755-9af0-32ba69fbf1e6 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:17.204728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1163/2306] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:51:17 2022] PUT /identity/v3/projects/581bcdfdd6e942d6a9a60d5f9ed5ef19/users/2780258f73ee4f5d8aeff7295d8f271c/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:51:17.214915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb6b9d-84a2-4103-9434-118a4c86ef96 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:17.215285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb6b9d-84a2-4103-9434-118a4c86ef96 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:17.215637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb6b9d-84a2-4103-9434-118a4c86ef96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:17.245806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-84cb6b9d-84a2-4103-9434-118a4c86ef96 None None] MFA Rules not processed for user `2780258f73ee4f5d8aeff7295d8f271c`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:17.247036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-84cb6b9d-84a2-4103-9434-118a4c86ef96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:17.318208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1144/2307] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:17.331270 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dfaea-64fb-4755-9af0-32ba69fbf1e6 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:17.333057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-867dfaea-64fb-4755-9af0-32ba69fbf1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:17.390800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dfaea-64fb-4755-9af0-32ba69fbf1e6 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:17.395403 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83f548-0785-4e81-a8d8-a65d69c6d698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:17.396081 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83f548-0785-4e81-a8d8-a65d69c6d698 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:17.396364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83f548-0785-4e81-a8d8-a65d69c6d698 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:17.396638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83f548-0785-4e81-a8d8-a65d69c6d698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:17.406074 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83f548-0785-4e81-a8d8-a65d69c6d698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:17.407528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83f548-0785-4e81-a8d8-a65d69c6d698 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:17.421896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1164/2308] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:17.453373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb6b9d-84a2-4103-9434-118a4c86ef96 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:17.519457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb6b9d-84a2-4103-9434-118a4c86ef96 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:17.525340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:17.526216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:17.526493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:17.526757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] PATH_INFO: `/v3/users/2780258f73ee4f5d8aeff7295d8f271c` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:17.528039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2780258f73ee4f5d8aeff7295d8f271c)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:17.530023 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:17.547987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:17.579599 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2780258f73ee4f5d8aeff7295d8f271c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:17.587787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2780258f73ee4f5d8aeff7295d8f271c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:17.588426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2780258f73ee4f5d8aeff7295d8f271c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:17.595465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2780258f73ee4f5d8aeff7295d8f271c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:17.611208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2780258f73ee4f5d8aeff7295d8f271c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:17.617977 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1145/2309] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:51:17 2022] DELETE /identity/v3/users/2780258f73ee4f5d8aeff7295d8f271c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:17.622592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a1277-9ca9-4797-b043-e595415c89c7 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:17.622879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a1277-9ca9-4797-b043-e595415c89c7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:17.623150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a1277-9ca9-4797-b043-e595415c89c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:17.641126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a73a1277-9ca9-4797-b043-e595415c89c7 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:17.642670 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a73a1277-9ca9-4797-b043-e595415c89c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:17.707968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1165/2310] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:17.717954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:17.719473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:17.788942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe7af8-14a8-48a2-8af7-ed60befb6431 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:17.792790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bbab2-4e5a-4861-a31a-747a7ade8d59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:17.793443 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bbab2-4e5a-4861-a31a-747a7ade8d59 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:17.793720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bbab2-4e5a-4861-a31a-747a7ade8d59 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:17.794013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bbab2-4e5a-4861-a31a-747a7ade8d59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:17.803723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bbab2-4e5a-4861-a31a-747a7ade8d59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:17.805302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bbab2-4e5a-4861-a31a-747a7ade8d59 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:17.816557 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1146/2311] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:18.099327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a1277-9ca9-4797-b043-e595415c89c7 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:18.101298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a73a1277-9ca9-4797-b043-e595415c89c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:18.119528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a1277-9ca9-4797-b043-e595415c89c7 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:18.125481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:18.126470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:18.126879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:18.127164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd admin admin] PATH_INFO: `/v3/projects/581bcdfdd6e942d6a9a60d5f9ed5ef19` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:18.128569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=581bcdfdd6e942d6a9a60d5f9ed5ef19)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:18.130592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:18.132306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:18.199927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '581bcdfdd6e942d6a9a60d5f9ed5ef19', 'request_id': 'req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:18.212898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '581bcdfdd6e942d6a9a60d5f9ed5ef19', 'request_id': 'req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:18.223343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '581bcdfdd6e942d6a9a60d5f9ed5ef19', 'request_id': 'req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:18.235616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd admin admin] The token cache is being invalidate because project 581bcdfdd6e942d6a9a60d5f9ed5ef19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:51:18.235616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:18.236456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-42b0a813-2b28-4e1e-aa07-dcbf8f6df6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:18.238040 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1166/2312] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:51:18 2022] DELETE /identity/v3/projects/581bcdfdd6e942d6a9a60d5f9ed5ef19 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:18.243818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f126a8f-9412-4e20-96d8-58c2a94fdcfb None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:18.244149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f126a8f-9412-4e20-96d8-58c2a94fdcfb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:18.244427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f126a8f-9412-4e20-96d8-58c2a94fdcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:18.267029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3f126a8f-9412-4e20-96d8-58c2a94fdcfb None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:18.269406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3f126a8f-9412-4e20-96d8-58c2a94fdcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:18.348057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1147/2313] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:18.355922 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b549945-bea1-491b-b877-30a592ab64da None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:18.356260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b549945-bea1-491b-b877-30a592ab64da None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:18.356547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b549945-bea1-491b-b877-30a592ab64da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:18.377280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-4b549945-bea1-491b-b877-30a592ab64da None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:18.378794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4b549945-bea1-491b-b877-30a592ab64da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:18.393445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1167/2314] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:18.398036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f126a8f-9412-4e20-96d8-58c2a94fdcfb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:18.411355 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f126a8f-9412-4e20-96d8-58c2a94fdcfb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:18.415727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1b9b7-69f0-4816-a3ca-4d8f44baa4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:18.416435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1b9b7-69f0-4816-a3ca-4d8f44baa4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:18.416715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1b9b7-69f0-4816-a3ca-4d8f44baa4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:18.417132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1b9b7-69f0-4816-a3ca-4d8f44baa4f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:18.417866 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1b9b7-69f0-4816-a3ca-4d8f44baa4f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:51:18.418171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1b9b7-69f0-4816-a3ca-4d8f44baa4f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:18.419859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1b9b7-69f0-4816-a3ca-4d8f44baa4f4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:18.435982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1148/2315] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:51:18.440442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b549945-bea1-491b-b877-30a592ab64da None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:18.451780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b549945-bea1-491b-b877-30a592ab64da None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:18.456821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30719b86-01e7-49d4-879c-4572added88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:18.457482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30719b86-01e7-49d4-879c-4572added88a admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:18.457764 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30719b86-01e7-49d4-879c-4572added88a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:18.458511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30719b86-01e7-49d4-879c-4572added88a admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:18.458759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30719b86-01e7-49d4-879c-4572added88a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:18.460706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30719b86-01e7-49d4-879c-4572added88a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:18.491064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1168/2316] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:51:18 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:51:18.495209 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1b9b7-69f0-4816-a3ca-4d8f44baa4f4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:18.510819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1b9b7-69f0-4816-a3ca-4d8f44baa4f4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:18.534963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:18.579094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7529a53-d06d-4b9f-afa1-091546612f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:18.579833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7529a53-d06d-4b9f-afa1-091546612f5b admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:18.580390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7529a53-d06d-4b9f-afa1-091546612f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:18.580721 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7529a53-d06d-4b9f-afa1-091546612f5b admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:18.581477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7529a53-d06d-4b9f-afa1-091546612f5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:18.583656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7529a53-d06d-4b9f-afa1-091546612f5b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:18.606497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1149/2317] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:51:18 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:51:18.610791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30719b86-01e7-49d4-879c-4572added88a admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:18.621537 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30719b86-01e7-49d4-879c-4572added88a admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:18.626527 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7532b73-71b0-4d80-9a6b-7964b7acb003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:18.627287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7532b73-71b0-4d80-9a6b-7964b7acb003 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:18.627638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7532b73-71b0-4d80-9a6b-7964b7acb003 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:18.627988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7532b73-71b0-4d80-9a6b-7964b7acb003 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:18.628908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7532b73-71b0-4d80-9a6b-7964b7acb003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:18.631164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7532b73-71b0-4d80-9a6b-7964b7acb003 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:18.645617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1169/2318] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:51:18.648077 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7529a53-d06d-4b9f-afa1-091546612f5b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:18.663224 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7529a53-d06d-4b9f-afa1-091546612f5b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:18.668591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3071f2a-9e13-4c43-8a57-e98353f23c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:18.669270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3071f2a-9e13-4c43-8a57-e98353f23c15 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:18.669590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3071f2a-9e13-4c43-8a57-e98353f23c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:18.669870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3071f2a-9e13-4c43-8a57-e98353f23c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:18.670604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3071f2a-9e13-4c43-8a57-e98353f23c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:18.672266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3071f2a-9e13-4c43-8a57-e98353f23c15 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:18.682565 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1150/2319] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:51:18.686923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7532b73-71b0-4d80-9a6b-7964b7acb003 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:18.698709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7532b73-71b0-4d80-9a6b-7964b7acb003 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:18.705230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804c64a-938c-4acd-bb3b-0f04e0f8aab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:18.706217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804c64a-938c-4acd-bb3b-0f04e0f8aab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:18.706491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804c64a-938c-4acd-bb3b-0f04e0f8aab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:18.706770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804c64a-938c-4acd-bb3b-0f04e0f8aab8 admin admin] PATH_INFO: `/v3/projects/99d9e85c63234c77889f856bf95288f5/users/46d5c3f98d924538a34c4b3a57fcac14/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:18.723413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804c64a-938c-4acd-bb3b-0f04e0f8aab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d9e85c63234c77889f856bf95288f5, user_id=46d5c3f98d924538a34c4b3a57fcac14, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:18.738127 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804c64a-938c-4acd-bb3b-0f04e0f8aab8 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:18.759455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1170/2320] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:51:18 2022] PUT /identity/v3/projects/99d9e85c63234c77889f856bf95288f5/users/46d5c3f98d924538a34c4b3a57fcac14/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:51:18.770531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68795b37-1031-4642-a76f-d0813d67e1a5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:18.771140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68795b37-1031-4642-a76f-d0813d67e1a5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:18.771574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68795b37-1031-4642-a76f-d0813d67e1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:18.796789 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-68795b37-1031-4642-a76f-d0813d67e1a5 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:18.798407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-68795b37-1031-4642-a76f-d0813d67e1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:18.862684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1151/2321] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:18.873388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804c64a-938c-4acd-bb3b-0f04e0f8aab8 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:18.875236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7804c64a-938c-4acd-bb3b-0f04e0f8aab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:18.935403 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804c64a-938c-4acd-bb3b-0f04e0f8aab8 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:18.940355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d608816-adc3-4d18-859e-0a99c020bb2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:18.941349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d608816-adc3-4d18-859e-0a99c020bb2f service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:18.941848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d608816-adc3-4d18-859e-0a99c020bb2f service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:18.942279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d608816-adc3-4d18-859e-0a99c020bb2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:18.955325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d608816-adc3-4d18-859e-0a99c020bb2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:18.960117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d608816-adc3-4d18-859e-0a99c020bb2f service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:18.970258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1171/2322] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:24.726931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f65eb0-815f-43d9-9ef8-175fa5151052 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:24.727491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f65eb0-815f-43d9-9ef8-175fa5151052 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:24.727491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f65eb0-815f-43d9-9ef8-175fa5151052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:24.759410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b2f65eb0-815f-43d9-9ef8-175fa5151052 None None] MFA Rules not processed for user `46d5c3f98d924538a34c4b3a57fcac14`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:24.760676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b2f65eb0-815f-43d9-9ef8-175fa5151052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:24.844360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1152/2323] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:24.848941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d608816-adc3-4d18-859e-0a99c020bb2f service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:24.859025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d608816-adc3-4d18-859e-0a99c020bb2f service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:24.864093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7559e-48e6-4667-9e4b-f143757d13dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:24.864878 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7559e-48e6-4667-9e4b-f143757d13dc admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:24.865268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7559e-48e6-4667-9e4b-f143757d13dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:24.870066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7559e-48e6-4667-9e4b-f143757d13dc admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:24.870066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7559e-48e6-4667-9e4b-f143757d13dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:24.870066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7559e-48e6-4667-9e4b-f143757d13dc admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:24.913744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1172/2324] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:51:24 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:51:24.917990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f65eb0-815f-43d9-9ef8-175fa5151052 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:24.932621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f65eb0-815f-43d9-9ef8-175fa5151052 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:25.001978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:25.034792 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddb780-dfd4-4746-8e93-e823a5bcf807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:25.035511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddb780-dfd4-4746-8e93-e823a5bcf807 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:25.035796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddb780-dfd4-4746-8e93-e823a5bcf807 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:25.036105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddb780-dfd4-4746-8e93-e823a5bcf807 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:25.036811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddb780-dfd4-4746-8e93-e823a5bcf807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:25.039182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddb780-dfd4-4746-8e93-e823a5bcf807 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:25.067606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1153/2325] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:51:24 2022] POST /identity/v3/users => generated 296 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:51:25.068910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7559e-48e6-4667-9e4b-f143757d13dc admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:25.079887 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7559e-48e6-4667-9e4b-f143757d13dc admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:25.085654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97702361-6891-4c28-9c53-d10c3c1b4b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:25.086455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97702361-6891-4c28-9c53-d10c3c1b4b0a admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:25.086743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97702361-6891-4c28-9c53-d10c3c1b4b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:25.087027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97702361-6891-4c28-9c53-d10c3c1b4b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:25.087805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97702361-6891-4c28-9c53-d10c3c1b4b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:25.089991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97702361-6891-4c28-9c53-d10c3c1b4b0a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:25.091570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:25.112428 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1173/2326] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:51:25.117653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddb780-dfd4-4746-8e93-e823a5bcf807 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:25.127946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddb780-dfd4-4746-8e93-e823a5bcf807 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:25.132527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6641115-44c8-491c-8f49-6d325b287319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:25.133422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6641115-44c8-491c-8f49-6d325b287319 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:25.133704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6641115-44c8-491c-8f49-6d325b287319 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:25.133992 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6641115-44c8-491c-8f49-6d325b287319 admin admin] PATH_INFO: `/v3/projects/2abeb9c4daa240b08266c6de955e4bba/users/1aace136bb1e4ba289d41b2328b083e7/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:25.149003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6641115-44c8-491c-8f49-6d325b287319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abeb9c4daa240b08266c6de955e4bba, user_id=1aace136bb1e4ba289d41b2328b083e7, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:25.151712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6641115-44c8-491c-8f49-6d325b287319 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:25.166147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1154/2327] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:51:25 2022] PUT /identity/v3/projects/2abeb9c4daa240b08266c6de955e4bba/users/1aace136bb1e4ba289d41b2328b083e7/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:25.170232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97702361-6891-4c28-9c53-d10c3c1b4b0a admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:25.181074 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97702361-6891-4c28-9c53-d10c3c1b4b0a admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:25.246805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c38121-61f7-4358-bcff-0fdb43c3fb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:25.247720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c38121-61f7-4358-bcff-0fdb43c3fb93 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:25.248187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c38121-61f7-4358-bcff-0fdb43c3fb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:25.248538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c38121-61f7-4358-bcff-0fdb43c3fb93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:25.249618 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c38121-61f7-4358-bcff-0fdb43c3fb93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:25.251292 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c38121-61f7-4358-bcff-0fdb43c3fb93 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:25.263403 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1174/2328] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:51:25.271637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6641115-44c8-491c-8f49-6d325b287319 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:25.283171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6641115-44c8-491c-8f49-6d325b287319 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:25.288269 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bf9e5-63f4-4a6c-8b7e-31e5d7016e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:25.289210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bf9e5-63f4-4a6c-8b7e-31e5d7016e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:25.289489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bf9e5-63f4-4a6c-8b7e-31e5d7016e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:25.289767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bf9e5-63f4-4a6c-8b7e-31e5d7016e3f admin admin] PATH_INFO: `/v3/projects/2abeb9c4daa240b08266c6de955e4bba/users/1aace136bb1e4ba289d41b2328b083e7/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:25.291937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bf9e5-63f4-4a6c-8b7e-31e5d7016e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2abeb9c4daa240b08266c6de955e4bba, user_id=1aace136bb1e4ba289d41b2328b083e7, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:25.294961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bf9e5-63f4-4a6c-8b7e-31e5d7016e3f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:25.307453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1155/2329] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:51:25 2022] PUT /identity/v3/projects/2abeb9c4daa240b08266c6de955e4bba/users/1aace136bb1e4ba289d41b2328b083e7/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:51:27.561107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c38121-61f7-4358-bcff-0fdb43c3fb93 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:27.563087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f5c38121-61f7-4358-bcff-0fdb43c3fb93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:27.603755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c38121-61f7-4358-bcff-0fdb43c3fb93 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:27.718999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:27.721298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:27.721707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:27.722080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] PATH_INFO: `/v3/users/b65d57d5e3184f519353144a9b599ab0` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:27.723710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65d57d5e3184f519353144a9b599ab0)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:27.726018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:27.844779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65d57d5e3184f519353144a9b599ab0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:27.846355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65d57d5e3184f519353144a9b599ab0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:27.861706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65d57d5e3184f519353144a9b599ab0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:27.870567 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65d57d5e3184f519353144a9b599ab0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:27.884985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65d57d5e3184f519353144a9b599ab0'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:27.898273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1175/2330] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:51:27 2022] DELETE /identity/v3/users/b65d57d5e3184f519353144a9b599ab0 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:27.903104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659b64-555f-4bdf-add6-1aa53be6c8dd None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:27.903806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659b64-555f-4bdf-add6-1aa53be6c8dd None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:27.903806 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659b64-555f-4bdf-add6-1aa53be6c8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:27.929733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-57659b64-555f-4bdf-add6-1aa53be6c8dd None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:27.931691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-57659b64-555f-4bdf-add6-1aa53be6c8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:28.031244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1156/2331] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:28.038468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:28.041094 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:28.069829 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:28.150389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f86d5c-4aa1-49c7-95be-3949462586b4 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:28.155784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12100b-b7bd-497c-8521-d4a469e82207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:28.156518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12100b-b7bd-497c-8521-d4a469e82207 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:28.156802 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12100b-b7bd-497c-8521-d4a469e82207 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:28.157078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12100b-b7bd-497c-8521-d4a469e82207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:28.169285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12100b-b7bd-497c-8521-d4a469e82207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:28.171124 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12100b-b7bd-497c-8521-d4a469e82207 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:28.235187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1176/2332] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:29.326388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57659b64-555f-4bdf-add6-1aa53be6c8dd None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:29.327943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-57659b64-555f-4bdf-add6-1aa53be6c8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:29.345383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57659b64-555f-4bdf-add6-1aa53be6c8dd None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:29.361813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034a4f9-0d5d-475d-b23a-30d101ec502f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:29.362969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034a4f9-0d5d-475d-b23a-30d101ec502f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:29.363815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034a4f9-0d5d-475d-b23a-30d101ec502f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:29.364276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034a4f9-0d5d-475d-b23a-30d101ec502f admin admin] PATH_INFO: `/v3/projects/5efebefbba7c4d0ba6f28e19eca1a371` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:29.366486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034a4f9-0d5d-475d-b23a-30d101ec502f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5efebefbba7c4d0ba6f28e19eca1a371)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:29.370477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034a4f9-0d5d-475d-b23a-30d101ec502f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:29.464206 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8034a4f9-0d5d-475d-b23a-30d101ec502f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5efebefbba7c4d0ba6f28e19eca1a371', 'request_id': 'req-8034a4f9-0d5d-475d-b23a-30d101ec502f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:29.479094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8034a4f9-0d5d-475d-b23a-30d101ec502f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5efebefbba7c4d0ba6f28e19eca1a371', 'request_id': 'req-8034a4f9-0d5d-475d-b23a-30d101ec502f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:29.488773 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8034a4f9-0d5d-475d-b23a-30d101ec502f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5efebefbba7c4d0ba6f28e19eca1a371', 'request_id': 'req-8034a4f9-0d5d-475d-b23a-30d101ec502f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:29.508845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8034a4f9-0d5d-475d-b23a-30d101ec502f admin admin] The token cache is being invalidate because project 5efebefbba7c4d0ba6f28e19eca1a371 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:51:29.509141 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8034a4f9-0d5d-475d-b23a-30d101ec502f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:29.509853 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8034a4f9-0d5d-475d-b23a-30d101ec502f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:29.511438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1157/2333] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:51:29 2022] DELETE /identity/v3/projects/5efebefbba7c4d0ba6f28e19eca1a371 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:29.516707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7a5fc-0a4f-49c1-b35b-88e5890346b8 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:29.518107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7a5fc-0a4f-49c1-b35b-88e5890346b8 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:29.518370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7a5fc-0a4f-49c1-b35b-88e5890346b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:29.539940 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-13e7a5fc-0a4f-49c1-b35b-88e5890346b8 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:29.541573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-13e7a5fc-0a4f-49c1-b35b-88e5890346b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:29.651543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1177/2334] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:51:29.659810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fec51-4d9b-45d5-bca8-bcf477eec00b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:29.660551 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fec51-4d9b-45d5-bca8-bcf477eec00b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:29.660931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fec51-4d9b-45d5-bca8-bcf477eec00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:29.700788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8a3fec51-4d9b-45d5-bca8-bcf477eec00b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:29.702046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8a3fec51-4d9b-45d5-bca8-bcf477eec00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:29.742937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1158/2335] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:29.746979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7a5fc-0a4f-49c1-b35b-88e5890346b8 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:29.764833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7a5fc-0a4f-49c1-b35b-88e5890346b8 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:29.778708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7583b7f-b82d-4cad-8877-01d7323fc59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:29.779513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7583b7f-b82d-4cad-8877-01d7323fc59d admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:29.779861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7583b7f-b82d-4cad-8877-01d7323fc59d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:29.780606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7583b7f-b82d-4cad-8877-01d7323fc59d admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:29.781071 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7583b7f-b82d-4cad-8877-01d7323fc59d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:51:29.781446 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7583b7f-b82d-4cad-8877-01d7323fc59d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:29.783210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7583b7f-b82d-4cad-8877-01d7323fc59d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:29.807584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1178/2336] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:51:29.812376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fec51-4d9b-45d5-bca8-bcf477eec00b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:29.819150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:29.832888 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fec51-4d9b-45d5-bca8-bcf477eec00b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:29.842254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdecf15-71d4-4ada-a1e5-c6922816b91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:29.843092 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdecf15-71d4-4ada-a1e5-c6922816b91e admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:29.843409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdecf15-71d4-4ada-a1e5-c6922816b91e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:29.843802 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdecf15-71d4-4ada-a1e5-c6922816b91e admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:29.844768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdecf15-71d4-4ada-a1e5-c6922816b91e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:29.846968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdecf15-71d4-4ada-a1e5-c6922816b91e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:29.881958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1159/2337] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:51:29 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:51:29.885758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7583b7f-b82d-4cad-8877-01d7323fc59d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:29.897728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7583b7f-b82d-4cad-8877-01d7323fc59d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:29.998271 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a53607-e111-47db-aa1e-bf464c8d76ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:29.999235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a53607-e111-47db-aa1e-bf464c8d76ee admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:29.999585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a53607-e111-47db-aa1e-bf464c8d76ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:29.999935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a53607-e111-47db-aa1e-bf464c8d76ee admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:30.000867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a53607-e111-47db-aa1e-bf464c8d76ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:30.002973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a53607-e111-47db-aa1e-bf464c8d76ee admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:30.030044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1179/2338] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:51:29 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:51:30.033581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdecf15-71d4-4ada-a1e5-c6922816b91e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:30.046759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdecf15-71d4-4ada-a1e5-c6922816b91e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:30.053534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4f68-e3f4-4e08-9abd-7a300e4f95cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:30.054496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4f68-e3f4-4e08-9abd-7a300e4f95cb admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:30.055302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4f68-e3f4-4e08-9abd-7a300e4f95cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:30.055302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4f68-e3f4-4e08-9abd-7a300e4f95cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:30.056119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a4f68-e3f4-4e08-9abd-7a300e4f95cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:30.057834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a4f68-e3f4-4e08-9abd-7a300e4f95cb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:30.073335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1160/2339] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:51:30.077273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a53607-e111-47db-aa1e-bf464c8d76ee admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:30.094968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a53607-e111-47db-aa1e-bf464c8d76ee admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:30.105442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce70582-d754-4c69-a4f6-a502af11a259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:30.106743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce70582-d754-4c69-a4f6-a502af11a259 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:30.107240 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce70582-d754-4c69-a4f6-a502af11a259 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:30.107751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce70582-d754-4c69-a4f6-a502af11a259 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:30.108893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce70582-d754-4c69-a4f6-a502af11a259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:30.110938 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce70582-d754-4c69-a4f6-a502af11a259 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:30.123229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1180/2340] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:51:30.127391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4f68-e3f4-4e08-9abd-7a300e4f95cb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:30.138378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4f68-e3f4-4e08-9abd-7a300e4f95cb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:30.145007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78478964-f838-4a56-a700-29e2b4109eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:30.146742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78478964-f838-4a56-a700-29e2b4109eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:30.147244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78478964-f838-4a56-a700-29e2b4109eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:30.148264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78478964-f838-4a56-a700-29e2b4109eb1 admin admin] PATH_INFO: `/v3/projects/cd2e5f7d1e2e4aa3a98bbbf89a6ec276/users/9e27cbb381174481ade02b228faa8fb8/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:30.173380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78478964-f838-4a56-a700-29e2b4109eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd2e5f7d1e2e4aa3a98bbbf89a6ec276, user_id=9e27cbb381174481ade02b228faa8fb8, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:30.176779 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78478964-f838-4a56-a700-29e2b4109eb1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:30.202320 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1161/2341] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:51:30 2022] PUT /identity/v3/projects/cd2e5f7d1e2e4aa3a98bbbf89a6ec276/users/9e27cbb381174481ade02b228faa8fb8/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:30.208240 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b550213-13ed-45fd-9546-f617f1be4179 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:30.208815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b550213-13ed-45fd-9546-f617f1be4179 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:30.209276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b550213-13ed-45fd-9546-f617f1be4179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:30.234476 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-3b550213-13ed-45fd-9546-f617f1be4179 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:30.235855 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3b550213-13ed-45fd-9546-f617f1be4179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:30.279720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:30.340598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1181/2342] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:30.351288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78478964-f838-4a56-a700-29e2b4109eb1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:30.353159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-78478964-f838-4a56-a700-29e2b4109eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:30.422442 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78478964-f838-4a56-a700-29e2b4109eb1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:30.426404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb6509-23db-4cca-b72c-0c431aed0c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:30.427213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb6509-23db-4cca-b72c-0c431aed0c44 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:30.427596 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb6509-23db-4cca-b72c-0c431aed0c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:30.427971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb6509-23db-4cca-b72c-0c431aed0c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:30.440826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb6509-23db-4cca-b72c-0c431aed0c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:30.443057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb6509-23db-4cca-b72c-0c431aed0c44 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:30.460330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1162/2343] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:31.248302 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239a7fb-fe86-492c-bb4b-985e19adda2a None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:31.248890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239a7fb-fe86-492c-bb4b-985e19adda2a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:31.248890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239a7fb-fe86-492c-bb4b-985e19adda2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:31.296594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-6239a7fb-fe86-492c-bb4b-985e19adda2a None None] MFA Rules not processed for user `9e27cbb381174481ade02b228faa8fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:31.299363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6239a7fb-fe86-492c-bb4b-985e19adda2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:31.444689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1182/2344] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:31.449863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb6509-23db-4cca-b72c-0c431aed0c44 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:31.451683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e8cb6509-23db-4cca-b72c-0c431aed0c44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:31.633838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb6509-23db-4cca-b72c-0c431aed0c44 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:31.638527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1714117-6d13-4e09-8c78-76ed206cca7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:31.639798 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1714117-6d13-4e09-8c78-76ed206cca7b service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:31.640372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1714117-6d13-4e09-8c78-76ed206cca7b service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:31.640829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1714117-6d13-4e09-8c78-76ed206cca7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:31.652264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1714117-6d13-4e09-8c78-76ed206cca7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:31.654273 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1714117-6d13-4e09-8c78-76ed206cca7b service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:31.668196 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1163/2345] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:32.195372 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878d628-d522-4eda-a2e4-75cc6c36b439 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:32.195672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878d628-d522-4eda-a2e4-75cc6c36b439 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:32.195948 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878d628-d522-4eda-a2e4-75cc6c36b439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:32.229599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-c878d628-d522-4eda-a2e4-75cc6c36b439 None None] MFA Rules not processed for user `1aace136bb1e4ba289d41b2328b083e7`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:32.231051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c878d628-d522-4eda-a2e4-75cc6c36b439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:32.337092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1183/2346] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:32.352381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1714117-6d13-4e09-8c78-76ed206cca7b service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:32.355118 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b1714117-6d13-4e09-8c78-76ed206cca7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:32.455079 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:32.476995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1714117-6d13-4e09-8c78-76ed206cca7b service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:32.482415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfde39-49ea-4332-b565-7c2b1973291d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:32.483088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfde39-49ea-4332-b565-7c2b1973291d service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:32.483369 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfde39-49ea-4332-b565-7c2b1973291d service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:32.483641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfde39-49ea-4332-b565-7c2b1973291d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:32.485459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2fcfde39-49ea-4332-b565-7c2b1973291d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:32.546869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfde39-49ea-4332-b565-7c2b1973291d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:32.548550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfde39-49ea-4332-b565-7c2b1973291d service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:32.558409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1164/2347] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:34.311137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a680fbf-7549-4b04-8dd9-345c2be4b1ce None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:34.311611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a680fbf-7549-4b04-8dd9-345c2be4b1ce None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:34.311736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a680fbf-7549-4b04-8dd9-345c2be4b1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:34.332443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2a680fbf-7549-4b04-8dd9-345c2be4b1ce None None] MFA Rules not processed for user `9e27cbb381174481ade02b228faa8fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:34.333845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2a680fbf-7549-4b04-8dd9-345c2be4b1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:34.362336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1184/2348] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:42.554386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=33 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:42.988238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0484f-c4ec-4294-a2ad-589a4180a163 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:42.989612 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0484f-c4ec-4294-a2ad-589a4180a163 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:42.990008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0484f-c4ec-4294-a2ad-589a4180a163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:43.029941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-eef0484f-c4ec-4294-a2ad-589a4180a163 None None] MFA Rules not processed for user `2ea01587de8f4d5584d1b12ef042d421`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:43.031849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-eef0484f-c4ec-4294-a2ad-589a4180a163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:43.113228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1165/2349] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:43.790944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd055f-35cb-4712-ad02-4a2233cac6ea None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:43.791700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd055f-35cb-4712-ad02-4a2233cac6ea None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:43.791700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd055f-35cb-4712-ad02-4a2233cac6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:43.816738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-5dfd055f-35cb-4712-ad02-4a2233cac6ea None None] MFA Rules not processed for user `4ac2825db0c649cab6dcc2e27fbaf80d`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:43.817954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5dfd055f-35cb-4712-ad02-4a2233cac6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:43.865327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:43.900777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1185/2350] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:53.104322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=25 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:53.894315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=12 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:57.666015 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0484f-c4ec-4294-a2ad-589a4180a163 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:57.667569 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-eef0484f-c4ec-4294-a2ad-589a4180a163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:57.683026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0484f-c4ec-4294-a2ad-589a4180a163 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:57.687713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:57.688638 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:57.688919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:57.689263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] PATH_INFO: `/v3/users/99d82e4f5e90442fae210efc2ba5e44f` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:57.690464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d82e4f5e90442fae210efc2ba5e44f)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:57.692437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:57.746457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d82e4f5e90442fae210efc2ba5e44f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:57.755315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d82e4f5e90442fae210efc2ba5e44f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:57.756019 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d82e4f5e90442fae210efc2ba5e44f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:57.763716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d82e4f5e90442fae210efc2ba5e44f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:57.778887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d82e4f5e90442fae210efc2ba5e44f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:57.785119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1166/2351] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:51:57 2022] DELETE /identity/v3/users/99d82e4f5e90442fae210efc2ba5e44f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:57.789553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334646fc-24e4-4ced-afde-925b713f9235 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:57.790391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334646fc-24e4-4ced-afde-925b713f9235 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:57.790656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334646fc-24e4-4ced-afde-925b713f9235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:57.808213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-334646fc-24e4-4ced-afde-925b713f9235 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:57.809441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-334646fc-24e4-4ced-afde-925b713f9235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:57.870562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1186/2352] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:57.880458 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:57.882020 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:57.939675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae47a2-835b-4b47-9c21-87634efe484f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:57.947310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c0723-bc84-4c09-b812-2069196d6cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:57.947992 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c0723-bc84-4c09-b812-2069196d6cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:57.948321 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c0723-bc84-4c09-b812-2069196d6cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:57.948611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c0723-bc84-4c09-b812-2069196d6cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:57.962491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c0723-bc84-4c09-b812-2069196d6cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:57.964572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c0723-bc84-4c09-b812-2069196d6cb1 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:57.974949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1167/2353] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:51:58.276991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334646fc-24e4-4ced-afde-925b713f9235 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:58.278868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-334646fc-24e4-4ced-afde-925b713f9235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:58.291346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334646fc-24e4-4ced-afde-925b713f9235 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:58.296053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf60bb8-c404-46a6-b479-fde963a36d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:58.297007 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf60bb8-c404-46a6-b479-fde963a36d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:58.297375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf60bb8-c404-46a6-b479-fde963a36d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:58.297716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf60bb8-c404-46a6-b479-fde963a36d8f admin admin] PATH_INFO: `/v3/projects/487d80530bdc47d68d3dce4a978ca6e9` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:58.299173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf60bb8-c404-46a6-b479-fde963a36d8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=487d80530bdc47d68d3dce4a978ca6e9)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:58.301319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf60bb8-c404-46a6-b479-fde963a36d8f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:58.352971 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-acf60bb8-c404-46a6-b479-fde963a36d8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '487d80530bdc47d68d3dce4a978ca6e9', 'request_id': 'req-acf60bb8-c404-46a6-b479-fde963a36d8f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:58.360854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-acf60bb8-c404-46a6-b479-fde963a36d8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '487d80530bdc47d68d3dce4a978ca6e9', 'request_id': 'req-acf60bb8-c404-46a6-b479-fde963a36d8f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:58.367744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-acf60bb8-c404-46a6-b479-fde963a36d8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '487d80530bdc47d68d3dce4a978ca6e9', 'request_id': 'req-acf60bb8-c404-46a6-b479-fde963a36d8f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:58.378106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-acf60bb8-c404-46a6-b479-fde963a36d8f admin admin] The token cache is being invalidate because project 487d80530bdc47d68d3dce4a978ca6e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:51:58.378390 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-acf60bb8-c404-46a6-b479-fde963a36d8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:58.379029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-acf60bb8-c404-46a6-b479-fde963a36d8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:51:58.380675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1187/2354] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:51:58 2022] DELETE /identity/v3/projects/487d80530bdc47d68d3dce4a978ca6e9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:58.385552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbfa98-2c08-4292-8efb-73126ce4df1d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:58.386028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbfa98-2c08-4292-8efb-73126ce4df1d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:58.386377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbfa98-2c08-4292-8efb-73126ce4df1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:58.405654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-bbcbfa98-2c08-4292-8efb-73126ce4df1d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:58.407067 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bbcbfa98-2c08-4292-8efb-73126ce4df1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:58.474163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1168/2355] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:58.481837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f3da1-7b16-4f24-8ec1-669c08b44313 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:58.482124 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f3da1-7b16-4f24-8ec1-669c08b44313 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:58.482403 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f3da1-7b16-4f24-8ec1-669c08b44313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:58.503522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-076f3da1-7b16-4f24-8ec1-669c08b44313 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:58.504732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-076f3da1-7b16-4f24-8ec1-669c08b44313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:58.514945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1188/2356] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:58.519569 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbfa98-2c08-4292-8efb-73126ce4df1d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:58.530201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbfa98-2c08-4292-8efb-73126ce4df1d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:58.535470 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e80df-1c62-49c5-8551-c25ccf456e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:58.536492 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e80df-1c62-49c5-8551-c25ccf456e80 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:58.536860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e80df-1c62-49c5-8551-c25ccf456e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:58.537239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e80df-1c62-49c5-8551-c25ccf456e80 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:58.538315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e80df-1c62-49c5-8551-c25ccf456e80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:51:58.538600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e80df-1c62-49c5-8551-c25ccf456e80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:58.540583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e80df-1c62-49c5-8551-c25ccf456e80 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:58.557978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1169/2357] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:51:58.562679 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f3da1-7b16-4f24-8ec1-669c08b44313 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:58.572776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f3da1-7b16-4f24-8ec1-669c08b44313 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:58.579310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529e5d1-f9d7-473a-a211-2683d5a145dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:58.580447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529e5d1-f9d7-473a-a211-2683d5a145dc admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:58.580738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529e5d1-f9d7-473a-a211-2683d5a145dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:58.581019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529e5d1-f9d7-473a-a211-2683d5a145dc admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:58.581732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529e5d1-f9d7-473a-a211-2683d5a145dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:58.583688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529e5d1-f9d7-473a-a211-2683d5a145dc admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:58.624057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e80df-1c62-49c5-8551-c25ccf456e80 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:58.625167 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1189/2358] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:51:58 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:51:58.632738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:58.640608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e80df-1c62-49c5-8551-c25ccf456e80 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:58.705252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319f073-414e-47f1-9ef8-dc774e4dec14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:58.705958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319f073-414e-47f1-9ef8-dc774e4dec14 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:58.706206 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319f073-414e-47f1-9ef8-dc774e4dec14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:58.706484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319f073-414e-47f1-9ef8-dc774e4dec14 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:58.707172 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319f073-414e-47f1-9ef8-dc774e4dec14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:58.709210 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319f073-414e-47f1-9ef8-dc774e4dec14 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:58.736294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1170/2359] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:51:58 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:51:58.740860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529e5d1-f9d7-473a-a211-2683d5a145dc admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:58.753147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529e5d1-f9d7-473a-a211-2683d5a145dc admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:58.757924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1329a3-cbdd-4e07-8d76-4c140640a6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:58.758606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1329a3-cbdd-4e07-8d76-4c140640a6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:58.758889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1329a3-cbdd-4e07-8d76-4c140640a6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:58.759168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1329a3-cbdd-4e07-8d76-4c140640a6c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:58.759911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1329a3-cbdd-4e07-8d76-4c140640a6c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:58.761562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1329a3-cbdd-4e07-8d76-4c140640a6c9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:58.772620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1190/2360] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:51:58.776932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319f073-414e-47f1-9ef8-dc774e4dec14 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:58.787499 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319f073-414e-47f1-9ef8-dc774e4dec14 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:58.793387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe734b6-e19c-4c27-a5c3-e8726a97a266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:58.794005 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe734b6-e19c-4c27-a5c3-e8726a97a266 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:58.794292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe734b6-e19c-4c27-a5c3-e8726a97a266 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:58.794566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe734b6-e19c-4c27-a5c3-e8726a97a266 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:58.795292 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe734b6-e19c-4c27-a5c3-e8726a97a266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:58.796988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe734b6-e19c-4c27-a5c3-e8726a97a266 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:58.807310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1171/2361] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:51:58.811313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1329a3-cbdd-4e07-8d76-4c140640a6c9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:58.821755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1329a3-cbdd-4e07-8d76-4c140640a6c9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:58.826912 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb77f5-ce91-4e6e-9065-545f357d66d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:58.827810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb77f5-ce91-4e6e-9065-545f357d66d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:58.828148 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb77f5-ce91-4e6e-9065-545f357d66d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:58.828428 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb77f5-ce91-4e6e-9065-545f357d66d2 admin admin] PATH_INFO: `/v3/projects/50a7c25e52784b18a45a576337af4440/users/71f7eda03abf4a2f97a8bdc054814b56/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:58.843384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb77f5-ce91-4e6e-9065-545f357d66d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a7c25e52784b18a45a576337af4440, user_id=71f7eda03abf4a2f97a8bdc054814b56, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:58.846169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb77f5-ce91-4e6e-9065-545f357d66d2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:58.861303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1191/2362] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:51:58 2022] PUT /identity/v3/projects/50a7c25e52784b18a45a576337af4440/users/71f7eda03abf4a2f97a8bdc054814b56/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:51:58.872716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574cfa-6abc-4b10-bea9-0f2b05fe7a2b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:58.873098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574cfa-6abc-4b10-bea9-0f2b05fe7a2b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:58.873459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69574cfa-6abc-4b10-bea9-0f2b05fe7a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:58.907314 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-69574cfa-6abc-4b10-bea9-0f2b05fe7a2b None None] MFA Rules not processed for user `71f7eda03abf4a2f97a8bdc054814b56`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:51:58.909094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-69574cfa-6abc-4b10-bea9-0f2b05fe7a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:58.976228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1172/2363] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:51:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:51:58.988085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb77f5-ce91-4e6e-9065-545f357d66d2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:51:58.990877 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e3fb77f5-ce91-4e6e-9065-545f357d66d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:51:59.014327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:51:59.054256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb77f5-ce91-4e6e-9065-545f357d66d2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:51:59.058422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae6498-dbf2-439c-bf9a-cdf119fdd589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:51:59.059088 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae6498-dbf2-439c-bf9a-cdf119fdd589 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:51:59.059372 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae6498-dbf2-439c-bf9a-cdf119fdd589 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:51:59.059648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae6498-dbf2-439c-bf9a-cdf119fdd589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:51:59.069981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae6498-dbf2-439c-bf9a-cdf119fdd589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:51:59.071464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae6498-dbf2-439c-bf9a-cdf119fdd589 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:51:59.081052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1192/2364] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:51:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:52:05.050303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6e2d3-a11c-4735-9b09-a56f394c2f06 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:05.050841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6e2d3-a11c-4735-9b09-a56f394c2f06 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:05.050841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6e2d3-a11c-4735-9b09-a56f394c2f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:05.068304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9fa6e2d3-a11c-4735-9b09-a56f394c2f06 None None] MFA Rules not processed for user `46d5c3f98d924538a34c4b3a57fcac14`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:05.069619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9fa6e2d3-a11c-4735-9b09-a56f394c2f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:05.129918 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1173/2365] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:52:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:52:09.076660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=17 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:52:13.862800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae6498-dbf2-439c-bf9a-cdf119fdd589 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:13.945764 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae6498-dbf2-439c-bf9a-cdf119fdd589 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:13.954032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:13.955543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:13.961888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:13.961888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] PATH_INFO: `/v3/users/71f7eda03abf4a2f97a8bdc054814b56` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:13.962518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f7eda03abf4a2f97a8bdc054814b56)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:13.967393 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:14.047756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f7eda03abf4a2f97a8bdc054814b56'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:14.048585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f7eda03abf4a2f97a8bdc054814b56'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:14.059861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f7eda03abf4a2f97a8bdc054814b56'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:14.069512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f7eda03abf4a2f97a8bdc054814b56'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:14.083875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f7eda03abf4a2f97a8bdc054814b56'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:14.091136 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1193/2366] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:52:13 2022] DELETE /identity/v3/users/71f7eda03abf4a2f97a8bdc054814b56 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:52:14.095760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f234c97-d8fe-4715-bd5d-c00bd234e6c7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:14.096294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f234c97-d8fe-4715-bd5d-c00bd234e6c7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:14.096375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f234c97-d8fe-4715-bd5d-c00bd234e6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:14.115586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4f234c97-d8fe-4715-bd5d-c00bd234e6c7 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:14.117603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4f234c97-d8fe-4715-bd5d-c00bd234e6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:14.129629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:52:14.195721 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1174/2367] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:52:14.213602 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:14.215239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:14.288355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bab755-8c2d-4d81-8abc-8d3104c5efe9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:14.292901 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5cd5de-9e69-42f3-814a-c2e968c42432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:14.293638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cd5de-9e69-42f3-814a-c2e968c42432 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:14.294066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cd5de-9e69-42f3-814a-c2e968c42432 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:14.294450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cd5de-9e69-42f3-814a-c2e968c42432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:14.304882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5cd5de-9e69-42f3-814a-c2e968c42432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:14.306510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5cd5de-9e69-42f3-814a-c2e968c42432 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:14.326437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1194/2368] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:52:14.610077 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f234c97-d8fe-4715-bd5d-c00bd234e6c7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:14.611711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4f234c97-d8fe-4715-bd5d-c00bd234e6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:14.649035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f234c97-d8fe-4715-bd5d-c00bd234e6c7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:14.657123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a605195-53ed-48ab-9ee9-c16af7211e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:14.658026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a605195-53ed-48ab-9ee9-c16af7211e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:14.658307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a605195-53ed-48ab-9ee9-c16af7211e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:14.658579 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a605195-53ed-48ab-9ee9-c16af7211e24 admin admin] PATH_INFO: `/v3/projects/50a7c25e52784b18a45a576337af4440` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:14.659808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a605195-53ed-48ab-9ee9-c16af7211e24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a7c25e52784b18a45a576337af4440)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:14.661813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a605195-53ed-48ab-9ee9-c16af7211e24 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:14.724205 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8a605195-53ed-48ab-9ee9-c16af7211e24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a7c25e52784b18a45a576337af4440', 'request_id': 'req-8a605195-53ed-48ab-9ee9-c16af7211e24'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:14.734607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8a605195-53ed-48ab-9ee9-c16af7211e24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a7c25e52784b18a45a576337af4440', 'request_id': 'req-8a605195-53ed-48ab-9ee9-c16af7211e24'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:14.741198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8a605195-53ed-48ab-9ee9-c16af7211e24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a7c25e52784b18a45a576337af4440', 'request_id': 'req-8a605195-53ed-48ab-9ee9-c16af7211e24'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:14.752991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8a605195-53ed-48ab-9ee9-c16af7211e24 admin admin] The token cache is being invalidate because project 50a7c25e52784b18a45a576337af4440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:52:14.753273 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8a605195-53ed-48ab-9ee9-c16af7211e24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:14.753915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8a605195-53ed-48ab-9ee9-c16af7211e24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:14.755504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1175/2369] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:52:14 2022] DELETE /identity/v3/projects/50a7c25e52784b18a45a576337af4440 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:52:14.760395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3525b42-cdca-4bb7-b877-8030c400752e None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:14.760680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3525b42-cdca-4bb7-b877-8030c400752e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:14.760949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3525b42-cdca-4bb7-b877-8030c400752e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:14.779684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a3525b42-cdca-4bb7-b877-8030c400752e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:14.780920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a3525b42-cdca-4bb7-b877-8030c400752e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:14.798972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:52:14.843913 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1195/2370] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:52:14.852672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb06e7e-0b09-4cfe-a4f9-a09cc6ab87be None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:14.852963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb06e7e-0b09-4cfe-a4f9-a09cc6ab87be None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:14.853236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb06e7e-0b09-4cfe-a4f9-a09cc6ab87be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:14.872726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-ebb06e7e-0b09-4cfe-a4f9-a09cc6ab87be None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:14.873932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ebb06e7e-0b09-4cfe-a4f9-a09cc6ab87be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:14.882809 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1176/2371] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:52:14.886756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3525b42-cdca-4bb7-b877-8030c400752e None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:14.898086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3525b42-cdca-4bb7-b877-8030c400752e None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:14.902697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79274e37-aacd-4e83-b3f8-1eb10f5cc582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:14.903411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79274e37-aacd-4e83-b3f8-1eb10f5cc582 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:14.903692 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79274e37-aacd-4e83-b3f8-1eb10f5cc582 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:14.903963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79274e37-aacd-4e83-b3f8-1eb10f5cc582 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:14.904717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79274e37-aacd-4e83-b3f8-1eb10f5cc582 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:52:14.905041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79274e37-aacd-4e83-b3f8-1eb10f5cc582 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:14.906656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79274e37-aacd-4e83-b3f8-1eb10f5cc582 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:14.918776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1196/2372] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:52:14.923156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb06e7e-0b09-4cfe-a4f9-a09cc6ab87be None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:14.932805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb06e7e-0b09-4cfe-a4f9-a09cc6ab87be None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:14.937588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69136f4f-5162-4d1c-a58f-d1bc27551484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:14.938278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69136f4f-5162-4d1c-a58f-d1bc27551484 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:14.938559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69136f4f-5162-4d1c-a58f-d1bc27551484 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:14.938833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69136f4f-5162-4d1c-a58f-d1bc27551484 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:14.939523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69136f4f-5162-4d1c-a58f-d1bc27551484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:14.941525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69136f4f-5162-4d1c-a58f-d1bc27551484 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:14.969052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1177/2373] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:52:14 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:52:14.972973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79274e37-aacd-4e83-b3f8-1eb10f5cc582 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:14.984144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79274e37-aacd-4e83-b3f8-1eb10f5cc582 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:15.080962 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d615802-0658-4be2-8276-5d42ced3b925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:15.081874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d615802-0658-4be2-8276-5d42ced3b925 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:15.082177 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d615802-0658-4be2-8276-5d42ced3b925 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:15.082455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d615802-0658-4be2-8276-5d42ced3b925 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:15.083288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d615802-0658-4be2-8276-5d42ced3b925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:15.085316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d615802-0658-4be2-8276-5d42ced3b925 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:15.110327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1197/2374] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:52:14 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:52:15.114154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69136f4f-5162-4d1c-a58f-d1bc27551484 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:15.126015 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69136f4f-5162-4d1c-a58f-d1bc27551484 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:15.131204 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e70c51-0b33-4bd4-8eca-c9452ad1eacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:15.131958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e70c51-0b33-4bd4-8eca-c9452ad1eacb admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:15.132431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e70c51-0b33-4bd4-8eca-c9452ad1eacb admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:15.132718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e70c51-0b33-4bd4-8eca-c9452ad1eacb admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:15.133629 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e70c51-0b33-4bd4-8eca-c9452ad1eacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:15.135366 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e70c51-0b33-4bd4-8eca-c9452ad1eacb admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:15.148759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1178/2375] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:52:15.153995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d615802-0658-4be2-8276-5d42ced3b925 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:15.165966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d615802-0658-4be2-8276-5d42ced3b925 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:15.173057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ad490-54a8-45d9-afa9-b6c02736a15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:15.173947 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ad490-54a8-45d9-afa9-b6c02736a15e admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:15.174648 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ad490-54a8-45d9-afa9-b6c02736a15e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:15.175056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ad490-54a8-45d9-afa9-b6c02736a15e admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:15.176093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ad490-54a8-45d9-afa9-b6c02736a15e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:15.178355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ad490-54a8-45d9-afa9-b6c02736a15e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:15.195834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e70c51-0b33-4bd4-8eca-c9452ad1eacb admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:15.196301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1198/2376] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:52:15.209965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e70c51-0b33-4bd4-8eca-c9452ad1eacb admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:15.215174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560867c-e5b3-46f4-bf90-fb361bcfbcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:15.216472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560867c-e5b3-46f4-bf90-fb361bcfbcd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:15.216693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560867c-e5b3-46f4-bf90-fb361bcfbcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:15.216971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560867c-e5b3-46f4-bf90-fb361bcfbcd8 admin admin] PATH_INFO: `/v3/projects/09f7e3cc59e14a0885a761c4be5e898f/users/2ca0aea94b944212bc47c30f28265de0/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:15.233850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560867c-e5b3-46f4-bf90-fb361bcfbcd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f7e3cc59e14a0885a761c4be5e898f, user_id=2ca0aea94b944212bc47c30f28265de0, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:15.236924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560867c-e5b3-46f4-bf90-fb361bcfbcd8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:15.252513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1179/2377] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:52:15 2022] PUT /identity/v3/projects/09f7e3cc59e14a0885a761c4be5e898f/users/2ca0aea94b944212bc47c30f28265de0/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:52:15.263751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f63922-ba21-4423-8541-420764fad36d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:15.263751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f63922-ba21-4423-8541-420764fad36d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:15.264070 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f63922-ba21-4423-8541-420764fad36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:15.294450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-22f63922-ba21-4423-8541-420764fad36d None None] MFA Rules not processed for user `2ca0aea94b944212bc47c30f28265de0`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:15.295887 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-22f63922-ba21-4423-8541-420764fad36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:15.384995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1199/2378] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:52:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:52:15.394226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560867c-e5b3-46f4-bf90-fb361bcfbcd8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:15.396433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7560867c-e5b3-46f4-bf90-fb361bcfbcd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:15.443444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:52:15.466748 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560867c-e5b3-46f4-bf90-fb361bcfbcd8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:15.477677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59487dfb-ac36-400c-aa06-f7eec72cc9e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:15.479003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59487dfb-ac36-400c-aa06-f7eec72cc9e3 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:15.480310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59487dfb-ac36-400c-aa06-f7eec72cc9e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:15.480756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59487dfb-ac36-400c-aa06-f7eec72cc9e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:15.498784 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59487dfb-ac36-400c-aa06-f7eec72cc9e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:15.500306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59487dfb-ac36-400c-aa06-f7eec72cc9e3 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:15.517319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1180/2379] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:52:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:52:17.717135 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f63922-ba21-4423-8541-420764fad36d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:17.719431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-22f63922-ba21-4423-8541-420764fad36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:17.741678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:52:17.797823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f63922-ba21-4423-8541-420764fad36d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:17.803037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e28be-a887-4449-b08d-8fdae69ec770 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:17.803703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e28be-a887-4449-b08d-8fdae69ec770 service placement] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:17.804092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e28be-a887-4449-b08d-8fdae69ec770 service placement] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:17.804347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e28be-a887-4449-b08d-8fdae69ec770 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:17.806157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d75e28be-a887-4449-b08d-8fdae69ec770 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:17.818536 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e28be-a887-4449-b08d-8fdae69ec770 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:17.820231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e28be-a887-4449-b08d-8fdae69ec770 service placement] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:17.830541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1200/2380] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:52:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:52:17.969953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e5321-f593-402a-b4de-371f7865e6dd None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:17.970515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e5321-f593-402a-b4de-371f7865e6dd None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:17.970515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e5321-f593-402a-b4de-371f7865e6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:17.989033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d82e5321-f593-402a-b4de-371f7865e6dd None None] MFA Rules not processed for user `2ca0aea94b944212bc47c30f28265de0`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:17.990251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d82e5321-f593-402a-b4de-371f7865e6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:17.998663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1181/2381] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:52:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:52:27.041985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf368d4d-89de-43d4-80b1-f4ebdf978433 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:27.041985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf368d4d-89de-43d4-80b1-f4ebdf978433 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:27.042429 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf368d4d-89de-43d4-80b1-f4ebdf978433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:27.061511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-bf368d4d-89de-43d4-80b1-f4ebdf978433 None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:27.062717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-bf368d4d-89de-43d4-80b1-f4ebdf978433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:27.130762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1201/2382] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:52:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:52:27.171371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e5321-f593-402a-b4de-371f7865e6dd None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:27.172869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d82e5321-f593-402a-b4de-371f7865e6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:27.246305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e5321-f593-402a-b4de-371f7865e6dd None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:27.252518 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e89ef1-3b2d-4190-9169-4efe3f2eae58 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:27.253175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e89ef1-3b2d-4190-9169-4efe3f2eae58 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:27.253461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e89ef1-3b2d-4190-9169-4efe3f2eae58 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:27.253747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e89ef1-3b2d-4190-9169-4efe3f2eae58 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:27.262936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e89ef1-3b2d-4190-9169-4efe3f2eae58 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:27.264570 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e89ef1-3b2d-4190-9169-4efe3f2eae58 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:27.273191 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1182/2383] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:52:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:52:37.123033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=49 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:52:37.269641 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=45 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:52:37.455736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf368d4d-89de-43d4-80b1-f4ebdf978433 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:37.458021 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-bf368d4d-89de-43d4-80b1-f4ebdf978433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:37.530719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf368d4d-89de-43d4-80b1-f4ebdf978433 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:37.544473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c855f-c75e-43b7-b4d8-73d32e61ae91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:37.546795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c855f-c75e-43b7-b4d8-73d32e61ae91 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:37.547769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c855f-c75e-43b7-b4d8-73d32e61ae91 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:37.547769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c855f-c75e-43b7-b4d8-73d32e61ae91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:37.550303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-005c855f-c75e-43b7-b4d8-73d32e61ae91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:37.637383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c855f-c75e-43b7-b4d8-73d32e61ae91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:37.639613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c855f-c75e-43b7-b4d8-73d32e61ae91 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:37.652320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1202/2384] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:52:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:52:43.821513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a50672-cdf0-4778-aff5-0dd3be33626e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:43.822115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a50672-cdf0-4778-aff5-0dd3be33626e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:43.822115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a50672-cdf0-4778-aff5-0dd3be33626e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:43.840122 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-86a50672-cdf0-4778-aff5-0dd3be33626e None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:43.841375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-86a50672-cdf0-4778-aff5-0dd3be33626e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:43.848798 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1183/2385] 10.222.0.20 () {58 vars in 967 bytes} [Tue Jul 12 03:52:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:52:43.861360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c855f-c75e-43b7-b4d8-73d32e61ae91 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:43.874036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c855f-c75e-43b7-b4d8-73d32e61ae91 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:43.890080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658485e-cbf2-4736-94e0-c32ef9ac4262 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:43.890959 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658485e-cbf2-4736-94e0-c32ef9ac4262 service swift] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:43.891261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658485e-cbf2-4736-94e0-c32ef9ac4262 service swift] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:43.892072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658485e-cbf2-4736-94e0-c32ef9ac4262 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:43.903883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4658485e-cbf2-4736-94e0-c32ef9ac4262 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:43.906573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4658485e-cbf2-4736-94e0-c32ef9ac4262 service swift] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:43.916638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1203/2386] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:52:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:52:45.979445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a50672-cdf0-4778-aff5-0dd3be33626e None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:45.981048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-86a50672-cdf0-4778-aff5-0dd3be33626e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:46.047598 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a50672-cdf0-4778-aff5-0dd3be33626e None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:46.052511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51104d6-b10c-413d-8217-5aa50debde9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:46.053405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51104d6-b10c-413d-8217-5aa50debde9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:46.053685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51104d6-b10c-413d-8217-5aa50debde9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:46.053958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51104d6-b10c-413d-8217-5aa50debde9f admin admin] PATH_INFO: `/v3/users/2ea01587de8f4d5584d1b12ef042d421` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:46.055259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51104d6-b10c-413d-8217-5aa50debde9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea01587de8f4d5584d1b12ef042d421)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:46.057308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51104d6-b10c-413d-8217-5aa50debde9f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:46.106778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b51104d6-b10c-413d-8217-5aa50debde9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea01587de8f4d5584d1b12ef042d421'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:46.116294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b51104d6-b10c-413d-8217-5aa50debde9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea01587de8f4d5584d1b12ef042d421'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:46.117753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b51104d6-b10c-413d-8217-5aa50debde9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea01587de8f4d5584d1b12ef042d421'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:46.124780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b51104d6-b10c-413d-8217-5aa50debde9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea01587de8f4d5584d1b12ef042d421'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:46.137642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-b51104d6-b10c-413d-8217-5aa50debde9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea01587de8f4d5584d1b12ef042d421'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:46.145846 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1184/2387] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:52:45 2022] DELETE /identity/v3/users/2ea01587de8f4d5584d1b12ef042d421 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:52:46.351872 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658485e-cbf2-4736-94e0-c32ef9ac4262 service swift] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:46.367510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658485e-cbf2-4736-94e0-c32ef9ac4262 service swift] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:46.428289 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:46.429173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:46.429457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:46.429740 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] PATH_INFO: `/v3/projects/3e33bfd157c64f8688b9720d80a3a8af` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:46.430942 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e33bfd157c64f8688b9720d80a3a8af)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:46.432934 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:46.480743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:52:46.491327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e33bfd157c64f8688b9720d80a3a8af', 'request_id': 'req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:46.500634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e33bfd157c64f8688b9720d80a3a8af', 'request_id': 'req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:46.507139 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e33bfd157c64f8688b9720d80a3a8af', 'request_id': 'req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:46.519797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] The token cache is being invalidate because project 3e33bfd157c64f8688b9720d80a3a8af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:52:46.520246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:46.521189 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:46.523582 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1204/2388] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:52:46 2022] DELETE /identity/v3/projects/3e33bfd157c64f8688b9720d80a3a8af => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:52:46.717280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b67c4f-d87d-44d3-81cf-35ff7553c832 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:46.717604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b67c4f-d87d-44d3-81cf-35ff7553c832 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:46.717944 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b67c4f-d87d-44d3-81cf-35ff7553c832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:46.737247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c7b67c4f-d87d-44d3-81cf-35ff7553c832 None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:46.738418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c7b67c4f-d87d-44d3-81cf-35ff7553c832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:46.803857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1185/2389] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:52:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:52:46.817569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:46.819115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:46.896384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db95b4b-29fd-44b3-9f6d-7398f4a3deee admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:46.902678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8db9a3-c4a0-49fb-8628-960d1400fe60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:46.903773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8db9a3-c4a0-49fb-8628-960d1400fe60 service swift] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:46.904398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8db9a3-c4a0-49fb-8628-960d1400fe60 service swift] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:46.904791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8db9a3-c4a0-49fb-8628-960d1400fe60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:46.914893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8db9a3-c4a0-49fb-8628-960d1400fe60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:46.916843 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8db9a3-c4a0-49fb-8628-960d1400fe60 service swift] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:46.925101 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1205/2390] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:52:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:52:47.496869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb8c22-c215-4684-b624-badb9854a536 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:47.497348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb8c22-c215-4684-b624-badb9854a536 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:47.497454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb8c22-c215-4684-b624-badb9854a536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:47.515838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-37bb8c22-c215-4684-b624-badb9854a536 None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:47.517447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-37bb8c22-c215-4684-b624-badb9854a536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:47.525029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1186/2391] 10.222.0.20 () {58 vars in 967 bytes} [Tue Jul 12 03:52:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:52:47.539420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8db9a3-c4a0-49fb-8628-960d1400fe60 service swift] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:47.560903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8db9a3-c4a0-49fb-8628-960d1400fe60 service swift] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:47.567709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753f0fb-18f3-442f-af0f-8b3d8e3ed2b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:47.568407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753f0fb-18f3-442f-af0f-8b3d8e3ed2b4 service swift] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:47.568685 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753f0fb-18f3-442f-af0f-8b3d8e3ed2b4 service swift] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:47.568957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753f0fb-18f3-442f-af0f-8b3d8e3ed2b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:47.578757 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753f0fb-18f3-442f-af0f-8b3d8e3ed2b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:47.580438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f753f0fb-18f3-442f-af0f-8b3d8e3ed2b4 service swift] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:47.589129 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1206/2392] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:52:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:52:47.797941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d347958-5f82-4157-aa95-61b4bdc533cb None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:47.798233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d347958-5f82-4157-aa95-61b4bdc533cb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:47.798508 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d347958-5f82-4157-aa95-61b4bdc533cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:47.817121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9d347958-5f82-4157-aa95-61b4bdc533cb None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:47.818327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9d347958-5f82-4157-aa95-61b4bdc533cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:47.827285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1187/2393] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:52:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:52:47.839705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753f0fb-18f3-442f-af0f-8b3d8e3ed2b4 service swift] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:47.855513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753f0fb-18f3-442f-af0f-8b3d8e3ed2b4 service swift] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:47.861602 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06718aec-8661-4ede-841f-f6f0d534147a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:47.862247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06718aec-8661-4ede-841f-f6f0d534147a service swift] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:47.862534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06718aec-8661-4ede-841f-f6f0d534147a service swift] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:47.862814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06718aec-8661-4ede-841f-f6f0d534147a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:47.871536 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06718aec-8661-4ede-841f-f6f0d534147a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:47.873206 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06718aec-8661-4ede-841f-f6f0d534147a service swift] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:47.881440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1207/2394] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:52:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:52:48.208768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de8740-8ec9-4f32-b729-c20402a941e2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:48.209056 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de8740-8ec9-4f32-b729-c20402a941e2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:48.209360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de8740-8ec9-4f32-b729-c20402a941e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:48.217244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:52:48.230349 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-97de8740-8ec9-4f32-b729-c20402a941e2 None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:48.232378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-97de8740-8ec9-4f32-b729-c20402a941e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:48.241185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1188/2395] 10.222.0.20 () {58 vars in 967 bytes} [Tue Jul 12 03:52:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:52:48.254159 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06718aec-8661-4ede-841f-f6f0d534147a service swift] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:48.266970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06718aec-8661-4ede-841f-f6f0d534147a service swift] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:48.272550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d974c7-7e2e-4fc4-8ae2-f95977cfcb75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:48.273195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d974c7-7e2e-4fc4-8ae2-f95977cfcb75 service swift] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:48.273476 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d974c7-7e2e-4fc4-8ae2-f95977cfcb75 service swift] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:48.273747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d974c7-7e2e-4fc4-8ae2-f95977cfcb75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:48.282331 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d974c7-7e2e-4fc4-8ae2-f95977cfcb75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:48.283926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d974c7-7e2e-4fc4-8ae2-f95977cfcb75 service swift] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:48.292038 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1208/2396] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:52:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:52:50.409890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de8740-8ec9-4f32-b729-c20402a941e2 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:50.411609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-97de8740-8ec9-4f32-b729-c20402a941e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:50.481663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de8740-8ec9-4f32-b729-c20402a941e2 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:50.486537 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:50.487447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:50.487737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:50.488028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] PATH_INFO: `/v3/users/2ca0aea94b944212bc47c30f28265de0` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:50.489296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca0aea94b944212bc47c30f28265de0)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:50.491268 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:50.540878 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca0aea94b944212bc47c30f28265de0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:50.550814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca0aea94b944212bc47c30f28265de0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:50.551537 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca0aea94b944212bc47c30f28265de0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:50.559399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca0aea94b944212bc47c30f28265de0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:50.571724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca0aea94b944212bc47c30f28265de0'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:50.578706 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1189/2397] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:52:50 2022] DELETE /identity/v3/users/2ca0aea94b944212bc47c30f28265de0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:52:50.584626 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f10fa-2219-4375-8416-d4e001e1c4e4 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:50.585137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f10fa-2219-4375-8416-d4e001e1c4e4 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:50.585230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f10fa-2219-4375-8416-d4e001e1c4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:50.606731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-6c8f10fa-2219-4375-8416-d4e001e1c4e4 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:52:50.607963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6c8f10fa-2219-4375-8416-d4e001e1c4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:50.678089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1209/2398] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:52:50.688054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:50.689601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:50.751150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3180a2-ac3e-45ac-90fc-39e3093548fe admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:50.755468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d99435-b73d-4dd8-91aa-8335694bdf15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:50.756259 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d99435-b73d-4dd8-91aa-8335694bdf15 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:50.756568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d99435-b73d-4dd8-91aa-8335694bdf15 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:50.756885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d99435-b73d-4dd8-91aa-8335694bdf15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:50.767619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d99435-b73d-4dd8-91aa-8335694bdf15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:50.769131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d99435-b73d-4dd8-91aa-8335694bdf15 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:50.779234 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1190/2399] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:52:50.963718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f10fa-2219-4375-8416-d4e001e1c4e4 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:52:50.966058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6c8f10fa-2219-4375-8416-d4e001e1c4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:52:50.978011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f10fa-2219-4375-8416-d4e001e1c4e4 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:52:50.982597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77ee82-6b4f-4230-b511-130cb5098c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:52:50.983478 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77ee82-6b4f-4230-b511-130cb5098c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:52:50.983760 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77ee82-6b4f-4230-b511-130cb5098c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:52:50.984063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77ee82-6b4f-4230-b511-130cb5098c4e admin admin] PATH_INFO: `/v3/projects/09f7e3cc59e14a0885a761c4be5e898f` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:52:50.985255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77ee82-6b4f-4230-b511-130cb5098c4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09f7e3cc59e14a0885a761c4be5e898f)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:52:50.987181 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77ee82-6b4f-4230-b511-130cb5098c4e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:52:50.999027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:52:51.035824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3b77ee82-6b4f-4230-b511-130cb5098c4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09f7e3cc59e14a0885a761c4be5e898f', 'request_id': 'req-3b77ee82-6b4f-4230-b511-130cb5098c4e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:51.044115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3b77ee82-6b4f-4230-b511-130cb5098c4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09f7e3cc59e14a0885a761c4be5e898f', 'request_id': 'req-3b77ee82-6b4f-4230-b511-130cb5098c4e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:51.050656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3b77ee82-6b4f-4230-b511-130cb5098c4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09f7e3cc59e14a0885a761c4be5e898f', 'request_id': 'req-3b77ee82-6b4f-4230-b511-130cb5098c4e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:51.062288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3b77ee82-6b4f-4230-b511-130cb5098c4e admin admin] The token cache is being invalidate because project 09f7e3cc59e14a0885a761c4be5e898f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:52:51.062581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3b77ee82-6b4f-4230-b511-130cb5098c4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:51.063298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3b77ee82-6b4f-4230-b511-130cb5098c4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:52:51.065110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1210/2400] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:52:50 2022] DELETE /identity/v3/projects/09f7e3cc59e14a0885a761c4be5e898f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:53:00.774721 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=80,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:53:01.059247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:53:25.066675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374b312-1686-4413-b262-e132210df626 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:53:25.067330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374b312-1686-4413-b262-e132210df626 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:53:25.067330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374b312-1686-4413-b262-e132210df626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:53:25.086220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7374b312-1686-4413-b262-e132210df626 None None] MFA Rules not processed for user `9e27cbb381174481ade02b228faa8fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:53:25.087455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7374b312-1686-4413-b262-e132210df626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:53:25.144314 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1191/2401] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:53:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:53:27.313150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1ee52-5cbc-4ed1-b11d-681fec81dc53 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:53:27.313748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1ee52-5cbc-4ed1-b11d-681fec81dc53 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:53:27.313748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1ee52-5cbc-4ed1-b11d-681fec81dc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:53:27.333682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-63c1ee52-5cbc-4ed1-b11d-681fec81dc53 None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:53:27.335123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-63c1ee52-5cbc-4ed1-b11d-681fec81dc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:53:27.394724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1211/2402] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:53:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:53:27.418008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374b312-1686-4413-b262-e132210df626 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:53:27.419433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7374b312-1686-4413-b262-e132210df626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:53:27.493617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374b312-1686-4413-b262-e132210df626 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:53:27.500788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881316db-29e9-4301-92d4-84d34acb6807 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:53:27.501493 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881316db-29e9-4301-92d4-84d34acb6807 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:53:27.501776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881316db-29e9-4301-92d4-84d34acb6807 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:53:27.502057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881316db-29e9-4301-92d4-84d34acb6807 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:53:27.510998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881316db-29e9-4301-92d4-84d34acb6807 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:53:27.512572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881316db-29e9-4301-92d4-84d34acb6807 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:53:27.521421 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1192/2403] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:53:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 03:53:37.387547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=25 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:53:37.517896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=57 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:53:44.138149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11612ebc-91f7-47a3-9a21-1d58b27e40d2 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:53:44.139064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11612ebc-91f7-47a3-9a21-1d58b27e40d2 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:53:44.139726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11612ebc-91f7-47a3-9a21-1d58b27e40d2 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:53:44.142022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1212/2404] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:53:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:53:44.148910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93087c-eca7-44bb-9556-54ed8cb3cd13 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:53:44.149472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93087c-eca7-44bb-9556-54ed8cb3cd13 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:53:44.149472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93087c-eca7-44bb-9556-54ed8cb3cd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:53:44.173450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7a93087c-eca7-44bb-9556-54ed8cb3cd13 None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:53:44.174746 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7a93087c-eca7-44bb-9556-54ed8cb3cd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:53:44.255344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1193/2405] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:53:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:53:44.268478 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11612ebc-91f7-47a3-9a21-1d58b27e40d2 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:53:44.270284 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-11612ebc-91f7-47a3-9a21-1d58b27e40d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:53:44.328838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11612ebc-91f7-47a3-9a21-1d58b27e40d2 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:53:44.332519 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538164f7-3a73-4047-a449-0a98077c3ec6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7297224dc73c4358b60f1b096c0fb5bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:53:44.333164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538164f7-3a73-4047-a449-0a98077c3ec6 service glance] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:53:44.333453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538164f7-3a73-4047-a449-0a98077c3ec6 service glance] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:53:44.333729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538164f7-3a73-4047-a449-0a98077c3ec6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:53:44.343960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538164f7-3a73-4047-a449-0a98077c3ec6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:53:44.345477 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538164f7-3a73-4047-a449-0a98077c3ec6 service glance] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:53:44.356294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1213/2406] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:53:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:53:44.425820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00885f9-c99b-4d9a-8d81-e26f2805b928 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:53:44.426110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00885f9-c99b-4d9a-8d81-e26f2805b928 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:53:44.426390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00885f9-c99b-4d9a-8d81-e26f2805b928 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:53:44.427818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1194/2407] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:53:44.432771 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db091341-8a16-4897-9c3b-fc1a551d66cf None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:53:44.433065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db091341-8a16-4897-9c3b-fc1a551d66cf None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:53:44.433346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db091341-8a16-4897-9c3b-fc1a551d66cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:53:44.462824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-db091341-8a16-4897-9c3b-fc1a551d66cf None None] MFA Rules not processed for user `139be21e7f644b5796d30158851ebd00`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:53:44.464066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-db091341-8a16-4897-9c3b-fc1a551d66cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:53:44.523096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1214/2408] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:53:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:53:44.535080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00885f9-c99b-4d9a-8d81-e26f2805b928 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:53:44.536899 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d00885f9-c99b-4d9a-8d81-e26f2805b928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:53:44.551677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00885f9-c99b-4d9a-8d81-e26f2805b928 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:53:44.556586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d526203d-b41e-4e05-93b8-af1945e9df26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:53:44.557305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526203d-b41e-4e05-93b8-af1945e9df26 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:53:44.557600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526203d-b41e-4e05-93b8-af1945e9df26 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:53:44.557896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526203d-b41e-4e05-93b8-af1945e9df26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:53:44.565885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d526203d-b41e-4e05-93b8-af1945e9df26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:53:44.567337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d526203d-b41e-4e05-93b8-af1945e9df26 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:53:44.575816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1195/2409] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:53:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:53:44.663765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4839e7-8e6e-43c3-b5cd-57518a75d379 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:53:44.664061 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4839e7-8e6e-43c3-b5cd-57518a75d379 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:53:44.664373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4839e7-8e6e-43c3-b5cd-57518a75d379 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:53:44.666113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1215/2410] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:53:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:53:44.672182 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba02b11-0d32-4dcf-a428-d8a7e4f628c6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:53:44.672470 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba02b11-0d32-4dcf-a428-d8a7e4f628c6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:53:44.672745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba02b11-0d32-4dcf-a428-d8a7e4f628c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:53:44.697872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dba02b11-0d32-4dcf-a428-d8a7e4f628c6 None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:53:44.699088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dba02b11-0d32-4dcf-a428-d8a7e4f628c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:53:44.710265 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1196/2411] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:53:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:53:44.721861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4839e7-8e6e-43c3-b5cd-57518a75d379 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:53:44.734904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4839e7-8e6e-43c3-b5cd-57518a75d379 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:53:44.739233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb71ff-57ad-4428-ba9c-1c23b2d15d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:53:44.739908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb71ff-57ad-4428-ba9c-1c23b2d15d55 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:53:44.740222 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb71ff-57ad-4428-ba9c-1c23b2d15d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:53:44.740503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb71ff-57ad-4428-ba9c-1c23b2d15d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:53:44.756209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb71ff-57ad-4428-ba9c-1c23b2d15d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:53:44.757683 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb71ff-57ad-4428-ba9c-1c23b2d15d55 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:53:44.769686 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1216/2412] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:53:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:53:54.692391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=40 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:53:54.764574 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=55 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:54:01.037470 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995cd8c-039a-4731-ae1a-e843c77414c2 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:54:01.037942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995cd8c-039a-4731-ae1a-e843c77414c2 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:54:01.038103 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995cd8c-039a-4731-ae1a-e843c77414c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:54:01.060108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1995cd8c-039a-4731-ae1a-e843c77414c2 None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:54:01.061335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1995cd8c-039a-4731-ae1a-e843c77414c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:01.068844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1197/2413] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:54:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:54:01.081937 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb71ff-57ad-4428-ba9c-1c23b2d15d55 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:54:01.094179 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb71ff-57ad-4428-ba9c-1c23b2d15d55 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:54:01.099737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852d981-3b54-4d3d-a80d-9cf6a1f86b89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:54:01.100491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852d981-3b54-4d3d-a80d-9cf6a1f86b89 service swift] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:54:01.100803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852d981-3b54-4d3d-a80d-9cf6a1f86b89 service swift] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:54:01.101146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852d981-3b54-4d3d-a80d-9cf6a1f86b89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:54:01.111387 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852d981-3b54-4d3d-a80d-9cf6a1f86b89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:54:01.113084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852d981-3b54-4d3d-a80d-9cf6a1f86b89 service swift] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:54:01.122336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1217/2414] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:54:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:54:05.942016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995cd8c-039a-4731-ae1a-e843c77414c2 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:54:05.945923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1995cd8c-039a-4731-ae1a-e843c77414c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:06.008912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995cd8c-039a-4731-ae1a-e843c77414c2 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:54:06.012819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5cc62-839a-4880-9942-b46d0e46faf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:54:06.013472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5cc62-839a-4880-9942-b46d0e46faf5 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:54:06.013751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5cc62-839a-4880-9942-b46d0e46faf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:54:06.014027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5cc62-839a-4880-9942-b46d0e46faf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:54:06.015914 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3cd5cc62-839a-4880-9942-b46d0e46faf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:06.030251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5cc62-839a-4880-9942-b46d0e46faf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:54:06.031736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5cc62-839a-4880-9942-b46d0e46faf5 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:54:06.045732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1198/2415] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:54:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:54:09.088185 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852d981-3b54-4d3d-a80d-9cf6a1f86b89 service swift] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:54:09.090455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1852d981-3b54-4d3d-a80d-9cf6a1f86b89 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:09.107742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852d981-3b54-4d3d-a80d-9cf6a1f86b89 service swift] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:54:09.112528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd84034-abbd-48bf-9a3b-f0a1174fdbea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:54:09.113198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd84034-abbd-48bf-9a3b-f0a1174fdbea service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:54:09.113484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd84034-abbd-48bf-9a3b-f0a1174fdbea service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:54:09.113763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd84034-abbd-48bf-9a3b-f0a1174fdbea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:54:09.115844 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1dd84034-abbd-48bf-9a3b-f0a1174fdbea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:09.133773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd84034-abbd-48bf-9a3b-f0a1174fdbea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:54:09.135378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd84034-abbd-48bf-9a3b-f0a1174fdbea service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:54:09.148973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1218/2416] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:54:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:54:16.038708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=29 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:54:17.429049 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5cc62-839a-4880-9942-b46d0e46faf5 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:54:17.431753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3cd5cc62-839a-4880-9942-b46d0e46faf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:17.452177 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5cc62-839a-4880-9942-b46d0e46faf5 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:54:17.459828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46b0e3-3c1e-403f-afbf-683bc8ae96db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:54:17.460524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46b0e3-3c1e-403f-afbf-683bc8ae96db service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:54:17.460813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46b0e3-3c1e-403f-afbf-683bc8ae96db service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:54:17.461096 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46b0e3-3c1e-403f-afbf-683bc8ae96db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:54:17.463283 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2e46b0e3-3c1e-403f-afbf-683bc8ae96db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:17.531647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46b0e3-3c1e-403f-afbf-683bc8ae96db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:54:17.534099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46b0e3-3c1e-403f-afbf-683bc8ae96db service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:54:17.543954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1199/2417] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:54:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:54:19.143603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=12 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:54:27.539310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=28 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:54:30.937668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd84034-abbd-48bf-9a3b-f0a1174fdbea service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:54:30.947026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd84034-abbd-48bf-9a3b-f0a1174fdbea service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:54:30.951566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7caa8a-01bd-4e27-8808-42924f4fd0eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:54:30.952548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7caa8a-01bd-4e27-8808-42924f4fd0eb service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:54:30.952831 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7caa8a-01bd-4e27-8808-42924f4fd0eb service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:54:30.953105 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7caa8a-01bd-4e27-8808-42924f4fd0eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:54:30.955164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ad7caa8a-01bd-4e27-8808-42924f4fd0eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:30.970245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7caa8a-01bd-4e27-8808-42924f4fd0eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:54:30.971765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7caa8a-01bd-4e27-8808-42924f4fd0eb service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:54:30.981528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1219/2418] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:54:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:54:32.618623 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25333948-c588-4dea-8a90-4d1e45305c2b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:54:32.619214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25333948-c588-4dea-8a90-4d1e45305c2b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:54:32.619311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25333948-c588-4dea-8a90-4d1e45305c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:54:32.641342 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-25333948-c588-4dea-8a90-4d1e45305c2b None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:54:32.642684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-25333948-c588-4dea-8a90-4d1e45305c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:32.650640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1200/2419] 10.222.0.20 () {58 vars in 967 bytes} [Tue Jul 12 03:54:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:54:32.674691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7caa8a-01bd-4e27-8808-42924f4fd0eb service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:54:32.687822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7caa8a-01bd-4e27-8808-42924f4fd0eb service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:54:32.692982 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140477b-baf1-46b0-852c-cd138ab86483 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:54:32.693636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140477b-baf1-46b0-852c-cd138ab86483 service swift] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:54:32.693922 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140477b-baf1-46b0-852c-cd138ab86483 service swift] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:54:32.694202 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140477b-baf1-46b0-852c-cd138ab86483 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:54:32.702944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140477b-baf1-46b0-852c-cd138ab86483 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:54:32.704664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140477b-baf1-46b0-852c-cd138ab86483 service swift] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:54:32.713188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1220/2420] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:54:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:54:42.636404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:54:42.709785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=12 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:54:50.844394 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25333948-c588-4dea-8a90-4d1e45305c2b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:54:50.866048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25333948-c588-4dea-8a90-4d1e45305c2b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:54:50.876353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63809a13-ba19-49bd-a9a4-30a5b4313042 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:54:50.876996 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63809a13-ba19-49bd-a9a4-30a5b4313042 service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:54:50.877281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63809a13-ba19-49bd-a9a4-30a5b4313042 service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:54:50.877555 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63809a13-ba19-49bd-a9a4-30a5b4313042 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:54:50.879831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-63809a13-ba19-49bd-a9a4-30a5b4313042 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:50.942311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63809a13-ba19-49bd-a9a4-30a5b4313042 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:54:50.943958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63809a13-ba19-49bd-a9a4-30a5b4313042 service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:54:50.953366 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1201/2421] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:54:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:54:59.344927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140477b-baf1-46b0-852c-cd138ab86483 service swift] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:54:59.346471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1140477b-baf1-46b0-852c-cd138ab86483 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:59.416571 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63809a13-ba19-49bd-a9a4-30a5b4313042 service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:54:59.418084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-63809a13-ba19-49bd-a9a4-30a5b4313042 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:59.429488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140477b-baf1-46b0-852c-cd138ab86483 service swift] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:54:59.436665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e258a3-bc41-467c-ab09-cff56b09249c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:54:59.437211 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e258a3-bc41-467c-ab09-cff56b09249c service placement] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:54:59.437293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e258a3-bc41-467c-ab09-cff56b09249c service placement] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:54:59.437666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e258a3-bc41-467c-ab09-cff56b09249c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:54:59.441065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-83e258a3-bc41-467c-ab09-cff56b09249c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:54:59.454349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e258a3-bc41-467c-ab09-cff56b09249c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:54:59.456025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e258a3-bc41-467c-ab09-cff56b09249c service placement] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:54:59.466769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1221/2422] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:54:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:54:59.502202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63809a13-ba19-49bd-a9a4-30a5b4313042 service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:54:59.507318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5fde1-a90f-4b4e-87c0-61070ac174f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:54:59.508665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5fde1-a90f-4b4e-87c0-61070ac174f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:54:59.509154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5fde1-a90f-4b4e-87c0-61070ac174f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:54:59.509615 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5fde1-a90f-4b4e-87c0-61070ac174f3 admin admin] PATH_INFO: `/v3/users/9e27cbb381174481ade02b228faa8fb8` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:54:59.511006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb5fde1-a90f-4b4e-87c0-61070ac174f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e27cbb381174481ade02b228faa8fb8)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:54:59.514016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb5fde1-a90f-4b4e-87c0-61070ac174f3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:54:59.567841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-deb5fde1-a90f-4b4e-87c0-61070ac174f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e27cbb381174481ade02b228faa8fb8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:54:59.575814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-deb5fde1-a90f-4b4e-87c0-61070ac174f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e27cbb381174481ade02b228faa8fb8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:54:59.576771 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-deb5fde1-a90f-4b4e-87c0-61070ac174f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e27cbb381174481ade02b228faa8fb8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:54:59.583830 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-deb5fde1-a90f-4b4e-87c0-61070ac174f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e27cbb381174481ade02b228faa8fb8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:54:59.596680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-deb5fde1-a90f-4b4e-87c0-61070ac174f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e27cbb381174481ade02b228faa8fb8'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:54:59.602942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1202/2423] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:54:59 2022] DELETE /identity/v3/users/9e27cbb381174481ade02b228faa8fb8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:54:59.785903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e258a3-bc41-467c-ab09-cff56b09249c service placement] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:54:59.796856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e258a3-bc41-467c-ab09-cff56b09249c service placement] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:54:59.986294 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b8cd4-71ea-489e-aefd-72e6deca668f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:54:59.987094 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b8cd4-71ea-489e-aefd-72e6deca668f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:54:59.987378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b8cd4-71ea-489e-aefd-72e6deca668f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:54:59.987624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b8cd4-71ea-489e-aefd-72e6deca668f admin admin] PATH_INFO: `/v3/projects/cd2e5f7d1e2e4aa3a98bbbf89a6ec276` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:54:59.988897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b8cd4-71ea-489e-aefd-72e6deca668f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd2e5f7d1e2e4aa3a98bbbf89a6ec276)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:54:59.990868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b8cd4-71ea-489e-aefd-72e6deca668f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:00.037684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6d3b8cd4-71ea-489e-aefd-72e6deca668f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd2e5f7d1e2e4aa3a98bbbf89a6ec276', 'request_id': 'req-6d3b8cd4-71ea-489e-aefd-72e6deca668f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:55:00.044862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6d3b8cd4-71ea-489e-aefd-72e6deca668f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd2e5f7d1e2e4aa3a98bbbf89a6ec276', 'request_id': 'req-6d3b8cd4-71ea-489e-aefd-72e6deca668f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:55:00.050929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6d3b8cd4-71ea-489e-aefd-72e6deca668f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd2e5f7d1e2e4aa3a98bbbf89a6ec276', 'request_id': 'req-6d3b8cd4-71ea-489e-aefd-72e6deca668f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:55:00.061161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6d3b8cd4-71ea-489e-aefd-72e6deca668f admin admin] The token cache is being invalidate because project cd2e5f7d1e2e4aa3a98bbbf89a6ec276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:55:00.061449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6d3b8cd4-71ea-489e-aefd-72e6deca668f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:55:00.062123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-6d3b8cd4-71ea-489e-aefd-72e6deca668f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:55:00.063639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1222/2424] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:54:59 2022] DELETE /identity/v3/projects/cd2e5f7d1e2e4aa3a98bbbf89a6ec276 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:55:00.069074 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90938ba9-8cb6-4d5d-bf50-66e442c1e63d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:00.069344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90938ba9-8cb6-4d5d-bf50-66e442c1e63d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:00.069618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90938ba9-8cb6-4d5d-bf50-66e442c1e63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:00.087835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-90938ba9-8cb6-4d5d-bf50-66e442c1e63d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:55:00.089074 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-90938ba9-8cb6-4d5d-bf50-66e442c1e63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:00.102520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:55:00.154533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1203/2425] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:55:00.162597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a272e7-5347-454b-8d6d-a20005b8e8cc None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:00.162891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a272e7-5347-454b-8d6d-a20005b8e8cc None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:00.163168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a272e7-5347-454b-8d6d-a20005b8e8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:00.181773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-98a272e7-5347-454b-8d6d-a20005b8e8cc None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:55:00.183006 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-98a272e7-5347-454b-8d6d-a20005b8e8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:00.191881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1223/2426] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:55:00.196511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90938ba9-8cb6-4d5d-bf50-66e442c1e63d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:00.207044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90938ba9-8cb6-4d5d-bf50-66e442c1e63d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:00.214998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927812e-f488-4134-b3a2-3fbd4675cd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:00.215947 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927812e-f488-4134-b3a2-3fbd4675cd2f admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:00.216350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927812e-f488-4134-b3a2-3fbd4675cd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:00.216713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927812e-f488-4134-b3a2-3fbd4675cd2f admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:00.217783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927812e-f488-4134-b3a2-3fbd4675cd2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:55:00.218201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927812e-f488-4134-b3a2-3fbd4675cd2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:00.220728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927812e-f488-4134-b3a2-3fbd4675cd2f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:00.235141 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1204/2427] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 03:55:00.239840 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a272e7-5347-454b-8d6d-a20005b8e8cc None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:00.249787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a272e7-5347-454b-8d6d-a20005b8e8cc None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:00.255436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bcefe-f014-4a82-8c22-a7a93c1a4c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:00.256148 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bcefe-f014-4a82-8c22-a7a93c1a4c42 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:00.256430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bcefe-f014-4a82-8c22-a7a93c1a4c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:00.256701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bcefe-f014-4a82-8c22-a7a93c1a4c42 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:00.257379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bcefe-f014-4a82-8c22-a7a93c1a4c42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:00.259341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bcefe-f014-4a82-8c22-a7a93c1a4c42 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:00.270213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:55:00.289944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1224/2428] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:55:00 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:55:00.294244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927812e-f488-4134-b3a2-3fbd4675cd2f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:00.304877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927812e-f488-4134-b3a2-3fbd4675cd2f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:00.362240 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f3fe1-20a6-434b-8025-a78b6e81b3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:00.362953 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f3fe1-20a6-434b-8025-a78b6e81b3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:00.363237 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f3fe1-20a6-434b-8025-a78b6e81b3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:00.363509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f3fe1-20a6-434b-8025-a78b6e81b3e9 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:00.364264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f3fe1-20a6-434b-8025-a78b6e81b3e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:00.366220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f3fe1-20a6-434b-8025-a78b6e81b3e9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:00.388351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1205/2429] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:55:00 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:55:00.392703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bcefe-f014-4a82-8c22-a7a93c1a4c42 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:00.403883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bcefe-f014-4a82-8c22-a7a93c1a4c42 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:00.408706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d76cc2-f7d5-4339-834f-7423446376d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:00.409369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d76cc2-f7d5-4339-834f-7423446376d1 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:00.409653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d76cc2-f7d5-4339-834f-7423446376d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:00.409925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d76cc2-f7d5-4339-834f-7423446376d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:00.410653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d76cc2-f7d5-4339-834f-7423446376d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:00.412299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d76cc2-f7d5-4339-834f-7423446376d1 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:00.422665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1225/2430] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:55:00.426816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f3fe1-20a6-434b-8025-a78b6e81b3e9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:00.436872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f3fe1-20a6-434b-8025-a78b6e81b3e9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:00.441498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0e8a1-881e-484c-b0db-fb368567aa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:00.442158 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0e8a1-881e-484c-b0db-fb368567aa92 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:00.442468 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0e8a1-881e-484c-b0db-fb368567aa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:00.442750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0e8a1-881e-484c-b0db-fb368567aa92 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:00.443477 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0e8a1-881e-484c-b0db-fb368567aa92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:00.445136 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0e8a1-881e-484c-b0db-fb368567aa92 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:00.454901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1206/2431] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:55:00.458905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d76cc2-f7d5-4339-834f-7423446376d1 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:00.470640 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d76cc2-f7d5-4339-834f-7423446376d1 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:00.478171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7997bc-cd33-4204-8299-769e55619b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:00.479062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7997bc-cd33-4204-8299-769e55619b77 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:00.479341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7997bc-cd33-4204-8299-769e55619b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:00.479611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7997bc-cd33-4204-8299-769e55619b77 admin admin] PATH_INFO: `/v3/projects/b38d912121c64d5e887dc21e354b3cfd/users/8701284390d345eabc027e05f9ac262b/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:00.496672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7997bc-cd33-4204-8299-769e55619b77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38d912121c64d5e887dc21e354b3cfd, user_id=8701284390d345eabc027e05f9ac262b, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:00.499385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7997bc-cd33-4204-8299-769e55619b77 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:00.513520 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1226/2432] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:55:00 2022] PUT /identity/v3/projects/b38d912121c64d5e887dc21e354b3cfd/users/8701284390d345eabc027e05f9ac262b/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:55:00.521653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1eb14-dc8f-4b24-822e-9eba6f3fb4b9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:00.522159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1eb14-dc8f-4b24-822e-9eba6f3fb4b9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:00.522591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1eb14-dc8f-4b24-822e-9eba6f3fb4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:00.541662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dee1eb14-dc8f-4b24-822e-9eba6f3fb4b9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:55:00.542886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dee1eb14-dc8f-4b24-822e-9eba6f3fb4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:00.610922 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1207/2433] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:55:00.620609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7997bc-cd33-4204-8299-769e55619b77 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:00.622499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ad7997bc-cd33-4204-8299-769e55619b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:00.681303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7997bc-cd33-4204-8299-769e55619b77 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:00.685162 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa3d84-209d-41c7-919d-395599f7e8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:00.685822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa3d84-209d-41c7-919d-395599f7e8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:00.686111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa3d84-209d-41c7-919d-395599f7e8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:00.686388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa3d84-209d-41c7-919d-395599f7e8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:00.696550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa3d84-209d-41c7-919d-395599f7e8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:00.698004 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa3d84-209d-41c7-919d-395599f7e8d8 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:00.708138 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1227/2434] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:55:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:55:04.983013 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38488dc3-eb84-4581-bade-65b16b7b24fc None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:04.983461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38488dc3-eb84-4581-bade-65b16b7b24fc None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:04.983565 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38488dc3-eb84-4581-bade-65b16b7b24fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:05.022772 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-38488dc3-eb84-4581-bade-65b16b7b24fc None None] MFA Rules not processed for user `8701284390d345eabc027e05f9ac262b`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:55:05.023938 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-38488dc3-eb84-4581-bade-65b16b7b24fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:05.036723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:55:05.106277 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1208/2435] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:55:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:55:05.110776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa3d84-209d-41c7-919d-395599f7e8d8 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:05.122296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa3d84-209d-41c7-919d-395599f7e8d8 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:05.127559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9f521-d866-4d80-8904-e8af4cce6029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:05.128565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9f521-d866-4d80-8904-e8af4cce6029 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:05.129032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9f521-d866-4d80-8904-e8af4cce6029 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:05.129642 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9f521-d866-4d80-8904-e8af4cce6029 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:05.130934 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9f521-d866-4d80-8904-e8af4cce6029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:05.134939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9f521-d866-4d80-8904-e8af4cce6029 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:05.166213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1228/2436] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:55:05 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:55:05.170189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38488dc3-eb84-4581-bade-65b16b7b24fc None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:05.182427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38488dc3-eb84-4581-bade-65b16b7b24fc None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:05.253755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0783f1-3b7e-4fff-afa6-cb275611574d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:05.254444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0783f1-3b7e-4fff-afa6-cb275611574d admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:05.254728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0783f1-3b7e-4fff-afa6-cb275611574d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:05.255003 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0783f1-3b7e-4fff-afa6-cb275611574d admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:05.255801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0783f1-3b7e-4fff-afa6-cb275611574d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:05.257831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0783f1-3b7e-4fff-afa6-cb275611574d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:05.283029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1209/2437] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:55:05 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:55:05.284411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9f521-d866-4d80-8904-e8af4cce6029 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:05.295237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9f521-d866-4d80-8904-e8af4cce6029 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:05.300102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5fa80-bb37-4646-a1da-8987e7cb23ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:05.300860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5fa80-bb37-4646-a1da-8987e7cb23ec admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:05.301217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5fa80-bb37-4646-a1da-8987e7cb23ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:05.301562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5fa80-bb37-4646-a1da-8987e7cb23ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:05.302368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5fa80-bb37-4646-a1da-8987e7cb23ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:05.304101 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5fa80-bb37-4646-a1da-8987e7cb23ec admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:05.319292 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1229/2438] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:55:05.320535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0783f1-3b7e-4fff-afa6-cb275611574d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:05.330832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0783f1-3b7e-4fff-afa6-cb275611574d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:05.335456 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b929d-afab-498e-9043-94682b17c3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:05.336439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b929d-afab-498e-9043-94682b17c3fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:05.336713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b929d-afab-498e-9043-94682b17c3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:05.337032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b929d-afab-498e-9043-94682b17c3fe admin admin] PATH_INFO: `/v3/projects/e8a04014e6344f8fa9145b52d03679fa/users/f9eefffeaa3640219185515c6b3efbaf/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:05.353238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b929d-afab-498e-9043-94682b17c3fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a04014e6344f8fa9145b52d03679fa, user_id=f9eefffeaa3640219185515c6b3efbaf, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:05.355924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b929d-afab-498e-9043-94682b17c3fe admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:05.384473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1210/2439] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:55:05 2022] PUT /identity/v3/projects/e8a04014e6344f8fa9145b52d03679fa/users/f9eefffeaa3640219185515c6b3efbaf/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:55:05.388232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5fa80-bb37-4646-a1da-8987e7cb23ec admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:05.407311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5fa80-bb37-4646-a1da-8987e7cb23ec admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:05.468295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebafe52-d669-4f23-82b4-50515dfefb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:05.469140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebafe52-d669-4f23-82b4-50515dfefb54 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:05.469423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebafe52-d669-4f23-82b4-50515dfefb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:05.469695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebafe52-d669-4f23-82b4-50515dfefb54 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:05.470426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebafe52-d669-4f23-82b4-50515dfefb54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:05.472082 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebafe52-d669-4f23-82b4-50515dfefb54 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:05.483268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1230/2440] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:55:05.487498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b929d-afab-498e-9043-94682b17c3fe admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:05.499030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b929d-afab-498e-9043-94682b17c3fe admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:05.503891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641a0b0-7115-4a4b-9882-5256e767dc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:05.504916 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641a0b0-7115-4a4b-9882-5256e767dc94 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:05.505266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641a0b0-7115-4a4b-9882-5256e767dc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:05.505627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641a0b0-7115-4a4b-9882-5256e767dc94 admin admin] PATH_INFO: `/v3/projects/e8a04014e6344f8fa9145b52d03679fa/users/f9eefffeaa3640219185515c6b3efbaf/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:05.507763 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641a0b0-7115-4a4b-9882-5256e767dc94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a04014e6344f8fa9145b52d03679fa, user_id=f9eefffeaa3640219185515c6b3efbaf, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:05.510735 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641a0b0-7115-4a4b-9882-5256e767dc94 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:05.524267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1211/2441] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:55:05 2022] PUT /identity/v3/projects/e8a04014e6344f8fa9145b52d03679fa/users/f9eefffeaa3640219185515c6b3efbaf/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:55:09.968877 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a281f80-ad21-4ef6-86a5-7d9a1c4406ff None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:09.969356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a281f80-ad21-4ef6-86a5-7d9a1c4406ff None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:09.969470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a281f80-ad21-4ef6-86a5-7d9a1c4406ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:09.996306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-5a281f80-ad21-4ef6-86a5-7d9a1c4406ff None None] MFA Rules not processed for user `f9eefffeaa3640219185515c6b3efbaf`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:55:09.997516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5a281f80-ad21-4ef6-86a5-7d9a1c4406ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:10.007360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:55:10.068204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1231/2442] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:55:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:55:10.086353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641a0b0-7115-4a4b-9882-5256e767dc94 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:10.088317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d641a0b0-7115-4a4b-9882-5256e767dc94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:10.151454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641a0b0-7115-4a4b-9882-5256e767dc94 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:10.155671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229aed1-c5bf-4471-8fc1-0e10a343940e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7297224dc73c4358b60f1b096c0fb5bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:10.156181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229aed1-c5bf-4471-8fc1-0e10a343940e service glance] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:10.156483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229aed1-c5bf-4471-8fc1-0e10a343940e service glance] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:10.156983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229aed1-c5bf-4471-8fc1-0e10a343940e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:10.178926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:55:10.223816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229aed1-c5bf-4471-8fc1-0e10a343940e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:10.225350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229aed1-c5bf-4471-8fc1-0e10a343940e service glance] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:10.235331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1212/2443] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:55:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:55:10.341283 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cb745-7a7e-4b75-8a62-f5f56bcf3788 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:10.341824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cb745-7a7e-4b75-8a62-f5f56bcf3788 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:10.342259 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cb745-7a7e-4b75-8a62-f5f56bcf3788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:10.361213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a01cb745-7a7e-4b75-8a62-f5f56bcf3788 None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:55:10.362558 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a01cb745-7a7e-4b75-8a62-f5f56bcf3788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:10.423595 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1232/2444] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:55:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:55:10.435666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229aed1-c5bf-4471-8fc1-0e10a343940e service glance] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:10.437169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2229aed1-c5bf-4471-8fc1-0e10a343940e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:10.512427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229aed1-c5bf-4471-8fc1-0e10a343940e service glance] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:10.517530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496b4e8-df9e-48e2-a19e-f681c7f29166 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:10.518194 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496b4e8-df9e-48e2-a19e-f681c7f29166 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:10.518498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496b4e8-df9e-48e2-a19e-f681c7f29166 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:10.518774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496b4e8-df9e-48e2-a19e-f681c7f29166 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:10.528135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496b4e8-df9e-48e2-a19e-f681c7f29166 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:10.529768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496b4e8-df9e-48e2-a19e-f681c7f29166 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:10.538014 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1213/2445] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:55:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:55:19.293508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9180e62-13cc-4898-a1d8-25c7f09418eb None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:19.294097 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9180e62-13cc-4898-a1d8-25c7f09418eb None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:19.294097 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9180e62-13cc-4898-a1d8-25c7f09418eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:19.312911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-c9180e62-13cc-4898-a1d8-25c7f09418eb None None] MFA Rules not processed for user `8701284390d345eabc027e05f9ac262b`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:55:19.314463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c9180e62-13cc-4898-a1d8-25c7f09418eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:19.323511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1233/2446] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:55:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:55:19.576646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496b4e8-df9e-48e2-a19e-f681c7f29166 service swift] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:19.635964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496b4e8-df9e-48e2-a19e-f681c7f29166 service swift] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:19.639742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ac6ed-440d-450b-b235-d71cbe8bc3ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:19.640481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ac6ed-440d-450b-b235-d71cbe8bc3ed service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:19.640771 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ac6ed-440d-450b-b235-d71cbe8bc3ed service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:19.641048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ac6ed-440d-450b-b235-d71cbe8bc3ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:19.643008 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-656ac6ed-440d-450b-b235-d71cbe8bc3ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:19.653608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ac6ed-440d-450b-b235-d71cbe8bc3ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:19.655129 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ac6ed-440d-450b-b235-d71cbe8bc3ed service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:19.663444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1214/2447] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:55:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:55:20.000640 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9180e62-13cc-4898-a1d8-25c7f09418eb None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:20.002113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c9180e62-13cc-4898-a1d8-25c7f09418eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:20.077518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9180e62-13cc-4898-a1d8-25c7f09418eb None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:20.082575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e1129-3aad-4e7b-8923-194fe17efdb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:20.083222 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e1129-3aad-4e7b-8923-194fe17efdb7 service placement] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:20.083505 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e1129-3aad-4e7b-8923-194fe17efdb7 service placement] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:20.083778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e1129-3aad-4e7b-8923-194fe17efdb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:20.085644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ee2e1129-3aad-4e7b-8923-194fe17efdb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:20.096942 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e1129-3aad-4e7b-8923-194fe17efdb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:20.098575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e1129-3aad-4e7b-8923-194fe17efdb7 service placement] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:20.108424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1234/2448] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:55:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:55:24.313112 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7b89e-18d7-4e7c-a2d1-5f816958904e None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:24.313635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7b89e-18d7-4e7c-a2d1-5f816958904e None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:24.313635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7b89e-18d7-4e7c-a2d1-5f816958904e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:24.331685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4fd7b89e-18d7-4e7c-a2d1-5f816958904e None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:55:24.332943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4fd7b89e-18d7-4e7c-a2d1-5f816958904e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:24.340446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1215/2449] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:55:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:55:24.353751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e1129-3aad-4e7b-8923-194fe17efdb7 service placement] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:24.365794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e1129-3aad-4e7b-8923-194fe17efdb7 service placement] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:24.372830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c3eb5-eb8b-42fd-bbda-876ae4e881fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:24.373796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3eb5-eb8b-42fd-bbda-876ae4e881fd service swift] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:24.374012 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3eb5-eb8b-42fd-bbda-876ae4e881fd service swift] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:24.374287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3eb5-eb8b-42fd-bbda-876ae4e881fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:24.385221 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c3eb5-eb8b-42fd-bbda-876ae4e881fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:24.386913 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c3eb5-eb8b-42fd-bbda-876ae4e881fd service swift] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:24.396586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1235/2450] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:55:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:55:25.079996 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836daac9-17a7-43ef-84d5-9ca9aa6e502f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:25.080303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836daac9-17a7-43ef-84d5-9ca9aa6e502f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:25.080578 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836daac9-17a7-43ef-84d5-9ca9aa6e502f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:25.103617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-836daac9-17a7-43ef-84d5-9ca9aa6e502f None None] MFA Rules not processed for user `4ac2825db0c649cab6dcc2e27fbaf80d`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:55:25.104895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-836daac9-17a7-43ef-84d5-9ca9aa6e502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:25.160937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:55:25.174667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1216/2451] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:55:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:55:34.393450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=90 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:55:35.167753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:55:39.396465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c3eb5-eb8b-42fd-bbda-876ae4e881fd service swift] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:39.399636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2e4c3eb5-eb8b-42fd-bbda-876ae4e881fd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:39.480457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c3eb5-eb8b-42fd-bbda-876ae4e881fd service swift] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:39.486255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efe6a1-5c7f-4310-9b28-4439d09e83f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:39.486944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efe6a1-5c7f-4310-9b28-4439d09e83f5 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:39.487222 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efe6a1-5c7f-4310-9b28-4439d09e83f5 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:39.487498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efe6a1-5c7f-4310-9b28-4439d09e83f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:39.489371 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-41efe6a1-5c7f-4310-9b28-4439d09e83f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:39.512202 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efe6a1-5c7f-4310-9b28-4439d09e83f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:39.513302 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efe6a1-5c7f-4310-9b28-4439d09e83f5 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:39.538732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1236/2452] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:55:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:55:45.782209 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836daac9-17a7-43ef-84d5-9ca9aa6e502f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:45.797088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836daac9-17a7-43ef-84d5-9ca9aa6e502f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:45.802961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6521fc3-d126-4446-a33c-e23c8749ba6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:45.803680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521fc3-d126-4446-a33c-e23c8749ba6f service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:45.804023 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521fc3-d126-4446-a33c-e23c8749ba6f service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:45.804403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6521fc3-d126-4446-a33c-e23c8749ba6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:45.806227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e6521fc3-d126-4446-a33c-e23c8749ba6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:45.818852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6521fc3-d126-4446-a33c-e23c8749ba6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:45.820587 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6521fc3-d126-4446-a33c-e23c8749ba6f service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:45.831359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1217/2453] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:55:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:55:48.294304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efe6a1-5c7f-4310-9b28-4439d09e83f5 service nova] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:48.306149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efe6a1-5c7f-4310-9b28-4439d09e83f5 service nova] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:48.312628 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f1f7f-fe69-4b6d-bbfc-0dfad1b7864d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:48.313291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f1f7f-fe69-4b6d-bbfc-0dfad1b7864d service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:48.313576 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f1f7f-fe69-4b6d-bbfc-0dfad1b7864d service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:48.313836 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f1f7f-fe69-4b6d-bbfc-0dfad1b7864d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:48.316076 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-627f1f7f-fe69-4b6d-bbfc-0dfad1b7864d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:48.329768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f1f7f-fe69-4b6d-bbfc-0dfad1b7864d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:48.331425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f1f7f-fe69-4b6d-bbfc-0dfad1b7864d service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:48.345107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1237/2454] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:55:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:55:53.052988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6bc82-1449-42d9-a7fa-21191efa4516 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:53.053539 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6bc82-1449-42d9-a7fa-21191efa4516 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:53.053646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6bc82-1449-42d9-a7fa-21191efa4516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:53.075107 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-57f6bc82-1449-42d9-a7fa-21191efa4516 None None] MFA Rules not processed for user `46d5c3f98d924538a34c4b3a57fcac14`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:55:53.076409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-57f6bc82-1449-42d9-a7fa-21191efa4516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:53.162761 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1218/2455] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:55:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:55:54.400078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a521ea1-ed45-402a-85bf-9be650d38e47 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:54.400602 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a521ea1-ed45-402a-85bf-9be650d38e47 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:54.400602 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a521ea1-ed45-402a-85bf-9be650d38e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:54.419258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-9a521ea1-ed45-402a-85bf-9be650d38e47 None None] MFA Rules not processed for user `bf67549bf7e443c6b77eaf71fb6b709c`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:55:54.420530 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-9a521ea1-ed45-402a-85bf-9be650d38e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:54.484398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1238/2456] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:55:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:55:54.800415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6bc82-1449-42d9-a7fa-21191efa4516 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:55:54.802010 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-57f6bc82-1449-42d9-a7fa-21191efa4516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:54.814849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6bc82-1449-42d9-a7fa-21191efa4516 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:55:54.818324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aab9f6-6287-49e2-a46f-0b4bf4e765c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:55:54.818970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aab9f6-6287-49e2-a46f-0b4bf4e765c4 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:55:54.819252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aab9f6-6287-49e2-a46f-0b4bf4e765c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:55:54.819530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aab9f6-6287-49e2-a46f-0b4bf4e765c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:55:54.821346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d0aab9f6-6287-49e2-a46f-0b4bf4e765c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:55:54.830605 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aab9f6-6287-49e2-a46f-0b4bf4e765c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:55:54.832085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aab9f6-6287-49e2-a46f-0b4bf4e765c4 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:55:54.840031 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1219/2457] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:55:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:56:04.120806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a521ea1-ed45-402a-85bf-9be650d38e47 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:56:04.131609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a521ea1-ed45-402a-85bf-9be650d38e47 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:56:04.135941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42469c0b-e549-4775-b0c3-b91f12ee89dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:56:04.136608 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42469c0b-e549-4775-b0c3-b91f12ee89dc service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:56:04.136897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42469c0b-e549-4775-b0c3-b91f12ee89dc service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:56:04.137185 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42469c0b-e549-4775-b0c3-b91f12ee89dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:56:04.147155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42469c0b-e549-4775-b0c3-b91f12ee89dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:56:04.148646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42469c0b-e549-4775-b0c3-b91f12ee89dc service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:56:04.159485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1239/2458] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:56:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:56:04.836474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=37 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:56:05.123246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aab9f6-6287-49e2-a46f-0b4bf4e765c4 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:56:05.137735 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aab9f6-6287-49e2-a46f-0b4bf4e765c4 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:56:05.143583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7f0f6-97f7-42b4-a879-b2637824a0fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:56:05.144363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7f0f6-97f7-42b4-a879-b2637824a0fe service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:56:05.144711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7f0f6-97f7-42b4-a879-b2637824a0fe service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:56:05.145053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7f0f6-97f7-42b4-a879-b2637824a0fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:56:05.147400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bda7f0f6-97f7-42b4-a879-b2637824a0fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:56:05.212709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7f0f6-97f7-42b4-a879-b2637824a0fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:56:05.214640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7f0f6-97f7-42b4-a879-b2637824a0fe service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:56:05.225764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1220/2459] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:56:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:56:14.154209 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=71 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:56:15.221247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=28 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:57:10.269686 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42469c0b-e549-4775-b0c3-b91f12ee89dc service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:10.299457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42469c0b-e549-4775-b0c3-b91f12ee89dc service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:10.306231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd86c3-619c-41dd-bfe2-4a6047d95b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:10.307137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd86c3-619c-41dd-bfe2-4a6047d95b3d service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:10.307429 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd86c3-619c-41dd-bfe2-4a6047d95b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:10.307696 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd86c3-619c-41dd-bfe2-4a6047d95b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:10.310905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a9bd86c3-619c-41dd-bfe2-4a6047d95b3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:10.427907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd86c3-619c-41dd-bfe2-4a6047d95b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:10.429426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd86c3-619c-41dd-bfe2-4a6047d95b3d service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:10.449924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1240/2460] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:57:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:57:13.588166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7f0f6-97f7-42b4-a879-b2637824a0fe service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:13.644303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd86c3-619c-41dd-bfe2-4a6047d95b3d service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:13.645788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a9bd86c3-619c-41dd-bfe2-4a6047d95b3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:13.661107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd86c3-619c-41dd-bfe2-4a6047d95b3d service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:13.663189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7f0f6-97f7-42b4-a879-b2637824a0fe service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:13.665746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e3cc2-3771-40b1-a8ed-62a4af933292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:13.666634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e3cc2-3771-40b1-a8ed-62a4af933292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:13.666938 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e3cc2-3771-40b1-a8ed-62a4af933292 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:13.667427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e3cc2-3771-40b1-a8ed-62a4af933292 admin admin] PATH_INFO: `/v3/users/4ac2825db0c649cab6dcc2e27fbaf80d` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:13.668840 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e3cc2-3771-40b1-a8ed-62a4af933292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ac2825db0c649cab6dcc2e27fbaf80d)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:13.669896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdab6be-b7c1-49e2-897a-4b6f4a25ec81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:13.670756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdab6be-b7c1-49e2-897a-4b6f4a25ec81 service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:13.670879 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e3cc2-3771-40b1-a8ed-62a4af933292 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:13.671358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdab6be-b7c1-49e2-897a-4b6f4a25ec81 service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:13.671967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdab6be-b7c1-49e2-897a-4b6f4a25ec81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:13.674037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3bdab6be-b7c1-49e2-897a-4b6f4a25ec81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:13.706199 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdab6be-b7c1-49e2-897a-4b6f4a25ec81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:13.708965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdab6be-b7c1-49e2-897a-4b6f4a25ec81 service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:13.720634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1221/2461] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:57:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:57:13.747025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a58e3cc2-3771-40b1-a8ed-62a4af933292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ac2825db0c649cab6dcc2e27fbaf80d'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:13.747770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a58e3cc2-3771-40b1-a8ed-62a4af933292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ac2825db0c649cab6dcc2e27fbaf80d'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:13.759877 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a58e3cc2-3771-40b1-a8ed-62a4af933292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ac2825db0c649cab6dcc2e27fbaf80d'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:13.767288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a58e3cc2-3771-40b1-a8ed-62a4af933292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ac2825db0c649cab6dcc2e27fbaf80d'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:13.779984 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a58e3cc2-3771-40b1-a8ed-62a4af933292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ac2825db0c649cab6dcc2e27fbaf80d'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:13.781767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:57:13.787974 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1241/2462] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:57:13 2022] DELETE /identity/v3/users/4ac2825db0c649cab6dcc2e27fbaf80d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:57:13.994378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdab6be-b7c1-49e2-897a-4b6f4a25ec81 service placement] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:14.006077 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdab6be-b7c1-49e2-897a-4b6f4a25ec81 service placement] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:14.063379 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f6256-c0d7-4861-9fc4-0436e3349cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:14.064302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f6256-c0d7-4861-9fc4-0436e3349cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:14.064585 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f6256-c0d7-4861-9fc4-0436e3349cee admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:14.064860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f6256-c0d7-4861-9fc4-0436e3349cee admin admin] PATH_INFO: `/v3/projects/fd03cff94a1842d8811deafa0dce10f9` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:14.066038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f6256-c0d7-4861-9fc4-0436e3349cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd03cff94a1842d8811deafa0dce10f9)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:14.068114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f6256-c0d7-4861-9fc4-0436e3349cee admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:14.115795 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a37f6256-c0d7-4861-9fc4-0436e3349cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd03cff94a1842d8811deafa0dce10f9', 'request_id': 'req-a37f6256-c0d7-4861-9fc4-0436e3349cee'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:14.125657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a37f6256-c0d7-4861-9fc4-0436e3349cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd03cff94a1842d8811deafa0dce10f9', 'request_id': 'req-a37f6256-c0d7-4861-9fc4-0436e3349cee'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:14.131285 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a37f6256-c0d7-4861-9fc4-0436e3349cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd03cff94a1842d8811deafa0dce10f9', 'request_id': 'req-a37f6256-c0d7-4861-9fc4-0436e3349cee'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:14.142429 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a37f6256-c0d7-4861-9fc4-0436e3349cee admin admin] The token cache is being invalidate because project fd03cff94a1842d8811deafa0dce10f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:57:14.142720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a37f6256-c0d7-4861-9fc4-0436e3349cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:14.143362 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a37f6256-c0d7-4861-9fc4-0436e3349cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:14.144950 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1222/2463] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:57:13 2022] DELETE /identity/v3/projects/fd03cff94a1842d8811deafa0dce10f9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:57:14.150416 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e681bbe-9000-4ffa-a1b6-e4972a0b087d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:14.150731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e681bbe-9000-4ffa-a1b6-e4972a0b087d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:14.151009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e681bbe-9000-4ffa-a1b6-e4972a0b087d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:14.211085 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8e681bbe-9000-4ffa-a1b6-e4972a0b087d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:57:14.212326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8e681bbe-9000-4ffa-a1b6-e4972a0b087d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:14.271749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1242/2464] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:57:14.280747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642382fd-0706-4d3f-a1d3-00b8d28b9607 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:14.281034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642382fd-0706-4d3f-a1d3-00b8d28b9607 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:14.281333 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642382fd-0706-4d3f-a1d3-00b8d28b9607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:14.302142 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-642382fd-0706-4d3f-a1d3-00b8d28b9607 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:57:14.303368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-642382fd-0706-4d3f-a1d3-00b8d28b9607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:14.312766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1223/2465] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:57:14.317818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e681bbe-9000-4ffa-a1b6-e4972a0b087d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:14.331199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e681bbe-9000-4ffa-a1b6-e4972a0b087d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:14.338731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6fe38-a52f-4df1-99af-14bf3f5e0d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:14.340205 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6fe38-a52f-4df1-99af-14bf3f5e0d6b admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:14.340205 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6fe38-a52f-4df1-99af-14bf3f5e0d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:14.340205 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6fe38-a52f-4df1-99af-14bf3f5e0d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:14.340789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6fe38-a52f-4df1-99af-14bf3f5e0d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:57:14.341093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6fe38-a52f-4df1-99af-14bf3f5e0d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:14.342793 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a6fe38-a52f-4df1-99af-14bf3f5e0d6b admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:14.355643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1243/2466] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:57:14.360772 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642382fd-0706-4d3f-a1d3-00b8d28b9607 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:14.372178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642382fd-0706-4d3f-a1d3-00b8d28b9607 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:14.380185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6cc27-2533-4635-ab54-bdb980cd106f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:14.380850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6cc27-2533-4635-ab54-bdb980cd106f admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:14.381135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6cc27-2533-4635-ab54-bdb980cd106f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:14.381405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6cc27-2533-4635-ab54-bdb980cd106f admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:14.382097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6cc27-2533-4635-ab54-bdb980cd106f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:14.384042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6cc27-2533-4635-ab54-bdb980cd106f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:14.395276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:57:14.415607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1224/2467] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:57:14 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:57:14.421338 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6fe38-a52f-4df1-99af-14bf3f5e0d6b admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:14.431377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6fe38-a52f-4df1-99af-14bf3f5e0d6b admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:14.489991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c0eee-1e74-410e-b855-bd9efcc77114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:14.490688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0eee-1e74-410e-b855-bd9efcc77114 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:14.490967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0eee-1e74-410e-b855-bd9efcc77114 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:14.491281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c0eee-1e74-410e-b855-bd9efcc77114 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:14.491955 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c0eee-1e74-410e-b855-bd9efcc77114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:14.493908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c0eee-1e74-410e-b855-bd9efcc77114 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:14.517034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1244/2468] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:57:14 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:57:14.521333 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6cc27-2533-4635-ab54-bdb980cd106f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:14.532883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6cc27-2533-4635-ab54-bdb980cd106f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:14.538833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c1539-b3b5-4490-8a28-a607898b12e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:14.539533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c1539-b3b5-4490-8a28-a607898b12e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:14.539810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c1539-b3b5-4490-8a28-a607898b12e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:14.540114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c1539-b3b5-4490-8a28-a607898b12e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:14.540896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c1539-b3b5-4490-8a28-a607898b12e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:14.542498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c1539-b3b5-4490-8a28-a607898b12e1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:14.553242 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1225/2469] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:57:14.557195 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c0eee-1e74-410e-b855-bd9efcc77114 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:14.567712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c0eee-1e74-410e-b855-bd9efcc77114 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:14.573273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bde1c-a38b-4505-8c08-15376fa61c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:14.573943 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bde1c-a38b-4505-8c08-15376fa61c72 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:14.574233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bde1c-a38b-4505-8c08-15376fa61c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:14.574500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bde1c-a38b-4505-8c08-15376fa61c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:14.575204 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bde1c-a38b-4505-8c08-15376fa61c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:14.576842 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bde1c-a38b-4505-8c08-15376fa61c72 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:14.587097 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1245/2470] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:57:14.591509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c1539-b3b5-4490-8a28-a607898b12e1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:14.602272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c1539-b3b5-4490-8a28-a607898b12e1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:14.607873 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2ac0b-5f8e-4b33-8005-8e4324a01528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:14.608812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2ac0b-5f8e-4b33-8005-8e4324a01528 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:14.609097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2ac0b-5f8e-4b33-8005-8e4324a01528 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:14.609415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2ac0b-5f8e-4b33-8005-8e4324a01528 admin admin] PATH_INFO: `/v3/projects/03b40df2287441b991e9762f740fd48e/users/8d16939fa1ac4deb829820bf7dfae71f/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:14.625813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2ac0b-5f8e-4b33-8005-8e4324a01528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b40df2287441b991e9762f740fd48e, user_id=8d16939fa1ac4deb829820bf7dfae71f, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:14.628456 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2ac0b-5f8e-4b33-8005-8e4324a01528 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:14.646301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1226/2471] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:57:14 2022] PUT /identity/v3/projects/03b40df2287441b991e9762f740fd48e/users/8d16939fa1ac4deb829820bf7dfae71f/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:57:14.659107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1062107-3f91-4fc8-bbd1-b78af9febaf7 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:14.659654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1062107-3f91-4fc8-bbd1-b78af9febaf7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:14.659654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1062107-3f91-4fc8-bbd1-b78af9febaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:14.687074 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-c1062107-3f91-4fc8-bbd1-b78af9febaf7 None None] MFA Rules not processed for user `8d16939fa1ac4deb829820bf7dfae71f`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:57:14.688404 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c1062107-3f91-4fc8-bbd1-b78af9febaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:14.709146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:57:14.753492 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1246/2472] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:57:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:57:14.766245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2ac0b-5f8e-4b33-8005-8e4324a01528 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:14.768193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cbe2ac0b-5f8e-4b33-8005-8e4324a01528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:14.826995 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2ac0b-5f8e-4b33-8005-8e4324a01528 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:14.831243 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6ced0-58aa-4fb6-8079-321cf6ddf019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:14.831913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6ced0-58aa-4fb6-8079-321cf6ddf019 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:14.832246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6ced0-58aa-4fb6-8079-321cf6ddf019 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:14.832515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6ced0-58aa-4fb6-8079-321cf6ddf019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:14.841276 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6ced0-58aa-4fb6-8079-321cf6ddf019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:14.842725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6ced0-58aa-4fb6-8079-321cf6ddf019 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:14.851885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1227/2473] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:57:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:57:23.259233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1062107-3f91-4fc8-bbd1-b78af9febaf7 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:23.329439 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1062107-3f91-4fc8-bbd1-b78af9febaf7 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:23.334591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:23.335495 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:23.335777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:23.336048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] PATH_INFO: `/v3/users/8d16939fa1ac4deb829820bf7dfae71f` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:23.337276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d16939fa1ac4deb829820bf7dfae71f)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:23.339214 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:23.387633 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d16939fa1ac4deb829820bf7dfae71f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:23.388328 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d16939fa1ac4deb829820bf7dfae71f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:23.397665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d16939fa1ac4deb829820bf7dfae71f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:23.407898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d16939fa1ac4deb829820bf7dfae71f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:23.420557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d16939fa1ac4deb829820bf7dfae71f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:23.427821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1247/2474] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:57:23 2022] DELETE /identity/v3/users/8d16939fa1ac4deb829820bf7dfae71f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:57:23.432755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d229181-2738-4d07-a2e5-51608ba5dba1 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:23.433270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d229181-2738-4d07-a2e5-51608ba5dba1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:23.433270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d229181-2738-4d07-a2e5-51608ba5dba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:23.451639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5d229181-2738-4d07-a2e5-51608ba5dba1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:57:23.452915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5d229181-2738-4d07-a2e5-51608ba5dba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:23.531181 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1228/2475] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:57:23.541714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:23.543354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:23.596634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:57:23.610216 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a35f2f-a9fa-4d1f-9772-92c4fe9c627e admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:23.614240 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45151a-3657-4202-a5f6-00b9c1ad4a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:23.614925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45151a-3657-4202-a5f6-00b9c1ad4a52 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:23.615210 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45151a-3657-4202-a5f6-00b9c1ad4a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:23.615525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45151a-3657-4202-a5f6-00b9c1ad4a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:23.625091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45151a-3657-4202-a5f6-00b9c1ad4a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:23.626649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45151a-3657-4202-a5f6-00b9c1ad4a52 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:23.636118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1248/2476] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:57:23.930650 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d229181-2738-4d07-a2e5-51608ba5dba1 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:23.932170 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5d229181-2738-4d07-a2e5-51608ba5dba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:23.944765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d229181-2738-4d07-a2e5-51608ba5dba1 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:23.950301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863db9bf-5580-4311-9b09-6ecda736f2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:23.951187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863db9bf-5580-4311-9b09-6ecda736f2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:23.951465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863db9bf-5580-4311-9b09-6ecda736f2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:23.951737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863db9bf-5580-4311-9b09-6ecda736f2c6 admin admin] PATH_INFO: `/v3/projects/03b40df2287441b991e9762f740fd48e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:23.953051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863db9bf-5580-4311-9b09-6ecda736f2c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b40df2287441b991e9762f740fd48e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:23.955120 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863db9bf-5580-4311-9b09-6ecda736f2c6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:24.007224 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-863db9bf-5580-4311-9b09-6ecda736f2c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b40df2287441b991e9762f740fd48e', 'request_id': 'req-863db9bf-5580-4311-9b09-6ecda736f2c6'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:24.010813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:57:24.015614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-863db9bf-5580-4311-9b09-6ecda736f2c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b40df2287441b991e9762f740fd48e', 'request_id': 'req-863db9bf-5580-4311-9b09-6ecda736f2c6'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:24.022697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-863db9bf-5580-4311-9b09-6ecda736f2c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b40df2287441b991e9762f740fd48e', 'request_id': 'req-863db9bf-5580-4311-9b09-6ecda736f2c6'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:24.035016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-863db9bf-5580-4311-9b09-6ecda736f2c6 admin admin] The token cache is being invalidate because project 03b40df2287441b991e9762f740fd48e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:57:24.035246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-863db9bf-5580-4311-9b09-6ecda736f2c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:24.035895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-863db9bf-5580-4311-9b09-6ecda736f2c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:57:24.037488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1229/2477] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:57:23 2022] DELETE /identity/v3/projects/03b40df2287441b991e9762f740fd48e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:57:24.042537 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8e889-d535-43f6-bec0-2d7f920b923a None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.042827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8e889-d535-43f6-bec0-2d7f920b923a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.043107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8e889-d535-43f6-bec0-2d7f920b923a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.061147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-c7a8e889-d535-43f6-bec0-2d7f920b923a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:57:24.062345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c7a8e889-d535-43f6-bec0-2d7f920b923a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:24.129737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1249/2478] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:57:24.137931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531859dc-18c6-4aeb-a287-6c089851d6e8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.138226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531859dc-18c6-4aeb-a287-6c089851d6e8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.138501 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531859dc-18c6-4aeb-a287-6c089851d6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.167779 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-531859dc-18c6-4aeb-a287-6c089851d6e8 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:57:24.169684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-531859dc-18c6-4aeb-a287-6c089851d6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:24.180533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1230/2479] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:57:24.185288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8e889-d535-43f6-bec0-2d7f920b923a None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:24.195630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8e889-d535-43f6-bec0-2d7f920b923a None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:24.201888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1946d8-bd44-4f38-ac50-46362d1d9724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:24.202548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1946d8-bd44-4f38-ac50-46362d1d9724 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.202835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1946d8-bd44-4f38-ac50-46362d1d9724 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.203108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1946d8-bd44-4f38-ac50-46362d1d9724 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.204473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1946d8-bd44-4f38-ac50-46362d1d9724 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:57:24.204791 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1946d8-bd44-4f38-ac50-46362d1d9724 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:24.206409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1946d8-bd44-4f38-ac50-46362d1d9724 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:24.218568 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1250/2480] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:57:24.223048 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531859dc-18c6-4aeb-a287-6c089851d6e8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:24.232714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531859dc-18c6-4aeb-a287-6c089851d6e8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:24.238883 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe5e37-397e-47ce-becc-e311ed868b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:24.239538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe5e37-397e-47ce-becc-e311ed868b45 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.239801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe5e37-397e-47ce-becc-e311ed868b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.240131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe5e37-397e-47ce-becc-e311ed868b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.240815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe5e37-397e-47ce-becc-e311ed868b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:24.242720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe5e37-397e-47ce-becc-e311ed868b45 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:24.272518 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1231/2481] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:57:24 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:57:24.278261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1946d8-bd44-4f38-ac50-46362d1d9724 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:24.289343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1946d8-bd44-4f38-ac50-46362d1d9724 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:24.347665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1741438-8352-46a9-9097-80f406c6cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:24.348376 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1741438-8352-46a9-9097-80f406c6cc85 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.348666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1741438-8352-46a9-9097-80f406c6cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.348931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1741438-8352-46a9-9097-80f406c6cc85 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.349612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1741438-8352-46a9-9097-80f406c6cc85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:24.351528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1741438-8352-46a9-9097-80f406c6cc85 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:24.378707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1251/2482] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:57:24 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:57:24.380076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe5e37-397e-47ce-becc-e311ed868b45 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:24.391104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe5e37-397e-47ce-becc-e311ed868b45 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:24.398956 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61930460-a74f-4e29-9c44-0202f92e4828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:24.399644 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61930460-a74f-4e29-9c44-0202f92e4828 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.399928 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61930460-a74f-4e29-9c44-0202f92e4828 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.400229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61930460-a74f-4e29-9c44-0202f92e4828 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.400961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61930460-a74f-4e29-9c44-0202f92e4828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:24.402599 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61930460-a74f-4e29-9c44-0202f92e4828 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:24.413596 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1232/2483] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:57:24.417676 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1741438-8352-46a9-9097-80f406c6cc85 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:24.428441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1741438-8352-46a9-9097-80f406c6cc85 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:24.433664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df61e0-c8b0-4f5a-ac8a-af3ad2a09d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:24.434375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df61e0-c8b0-4f5a-ac8a-af3ad2a09d59 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.434632 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df61e0-c8b0-4f5a-ac8a-af3ad2a09d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.434905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df61e0-c8b0-4f5a-ac8a-af3ad2a09d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.435629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df61e0-c8b0-4f5a-ac8a-af3ad2a09d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:24.437296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df61e0-c8b0-4f5a-ac8a-af3ad2a09d59 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:24.448575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1252/2484] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:57:24.453026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61930460-a74f-4e29-9c44-0202f92e4828 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:24.463352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61930460-a74f-4e29-9c44-0202f92e4828 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:24.468509 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8ee1a-71b7-4f13-9c46-334651c03059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:24.469484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8ee1a-71b7-4f13-9c46-334651c03059 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.469766 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8ee1a-71b7-4f13-9c46-334651c03059 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.470042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8ee1a-71b7-4f13-9c46-334651c03059 admin admin] PATH_INFO: `/v3/projects/a13cd820ac8a4dcfb8cdcb19d967accf/users/dd570726eb504cb4b9382b0237bd04d1/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.487341 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8ee1a-71b7-4f13-9c46-334651c03059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a13cd820ac8a4dcfb8cdcb19d967accf, user_id=dd570726eb504cb4b9382b0237bd04d1, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:24.490053 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8ee1a-71b7-4f13-9c46-334651c03059 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:24.504105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1233/2485] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:57:24 2022] PUT /identity/v3/projects/a13cd820ac8a4dcfb8cdcb19d967accf/users/dd570726eb504cb4b9382b0237bd04d1/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:57:24.515647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f70e6-30bb-448a-9317-ed6855adc563 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.515958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f70e6-30bb-448a-9317-ed6855adc563 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.516286 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f70e6-30bb-448a-9317-ed6855adc563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.544680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-fb6f70e6-30bb-448a-9317-ed6855adc563 None None] MFA Rules not processed for user `dd570726eb504cb4b9382b0237bd04d1`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:57:24.545837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-fb6f70e6-30bb-448a-9317-ed6855adc563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:24.585604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:57:24.611155 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1253/2486] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:57:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:57:24.615794 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8ee1a-71b7-4f13-9c46-334651c03059 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:24.678908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8ee1a-71b7-4f13-9c46-334651c03059 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:24.684474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7d6e4-a8f7-425f-b7b9-d36034e0235e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:24.685143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7d6e4-a8f7-425f-b7b9-d36034e0235e admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.685433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7d6e4-a8f7-425f-b7b9-d36034e0235e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.685703 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7d6e4-a8f7-425f-b7b9-d36034e0235e admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.686386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7d6e4-a8f7-425f-b7b9-d36034e0235e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:24.688567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7d6e4-a8f7-425f-b7b9-d36034e0235e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:24.719033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1234/2487] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:57:24 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:57:24.724875 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f70e6-30bb-448a-9317-ed6855adc563 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:24.737172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f70e6-30bb-448a-9317-ed6855adc563 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:24.800711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d201d-8913-4f8c-a403-3f287a29c750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:24.801324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d201d-8913-4f8c-a403-3f287a29c750 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.801608 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d201d-8913-4f8c-a403-3f287a29c750 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.801883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d201d-8913-4f8c-a403-3f287a29c750 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.802585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d201d-8913-4f8c-a403-3f287a29c750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:24.804576 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d201d-8913-4f8c-a403-3f287a29c750 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:24.825727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1254/2488] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:57:24 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:57:24.829804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7d6e4-a8f7-425f-b7b9-d36034e0235e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:24.840207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7d6e4-a8f7-425f-b7b9-d36034e0235e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:24.844893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49638c-6684-445d-9517-784dba2e9833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:24.845557 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49638c-6684-445d-9517-784dba2e9833 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.845841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49638c-6684-445d-9517-784dba2e9833 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.846113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49638c-6684-445d-9517-784dba2e9833 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.846830 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49638c-6684-445d-9517-784dba2e9833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:24.848474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49638c-6684-445d-9517-784dba2e9833 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:24.858847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1235/2489] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:57:24.862979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d201d-8913-4f8c-a403-3f287a29c750 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:24.873601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d201d-8913-4f8c-a403-3f287a29c750 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:24.879539 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e938e3-d40a-4ee0-9b8c-09f6f8b3671a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:24.880474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e938e3-d40a-4ee0-9b8c-09f6f8b3671a admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.880757 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e938e3-d40a-4ee0-9b8c-09f6f8b3671a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.881035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e938e3-d40a-4ee0-9b8c-09f6f8b3671a admin admin] PATH_INFO: `/v3/projects/4ff88dd31f274cc7b1b89a0cb1afb0dd/users/aa2b34d63f6649c3ac287f791be3adf8/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.896526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e938e3-d40a-4ee0-9b8c-09f6f8b3671a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff88dd31f274cc7b1b89a0cb1afb0dd, user_id=aa2b34d63f6649c3ac287f791be3adf8, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:24.899600 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e938e3-d40a-4ee0-9b8c-09f6f8b3671a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:24.913679 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1255/2490] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:57:24 2022] PUT /identity/v3/projects/4ff88dd31f274cc7b1b89a0cb1afb0dd/users/aa2b34d63f6649c3ac287f791be3adf8/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:57:24.917978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49638c-6684-445d-9517-784dba2e9833 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:24.929011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49638c-6684-445d-9517-784dba2e9833 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:24.981305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:57:24.992692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338a554-553f-4422-8bc5-b51a86b3f140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:24.993375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338a554-553f-4422-8bc5-b51a86b3f140 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:24.993659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338a554-553f-4422-8bc5-b51a86b3f140 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:24.993934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338a554-553f-4422-8bc5-b51a86b3f140 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:24.994690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338a554-553f-4422-8bc5-b51a86b3f140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:24.996344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338a554-553f-4422-8bc5-b51a86b3f140 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:25.006624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1236/2491] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:57:25.011037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e938e3-d40a-4ee0-9b8c-09f6f8b3671a admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:25.022144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e938e3-d40a-4ee0-9b8c-09f6f8b3671a admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:25.027874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82949a0-7ed3-4b76-9794-80e203fd44db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:25.028764 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82949a0-7ed3-4b76-9794-80e203fd44db admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:25.029047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82949a0-7ed3-4b76-9794-80e203fd44db admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:25.029323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82949a0-7ed3-4b76-9794-80e203fd44db admin admin] PATH_INFO: `/v3/projects/4ff88dd31f274cc7b1b89a0cb1afb0dd/users/aa2b34d63f6649c3ac287f791be3adf8/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:25.031424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82949a0-7ed3-4b76-9794-80e203fd44db admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff88dd31f274cc7b1b89a0cb1afb0dd, user_id=aa2b34d63f6649c3ac287f791be3adf8, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:25.034732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82949a0-7ed3-4b76-9794-80e203fd44db admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:25.046470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1256/2492] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:57:25 2022] PUT /identity/v3/projects/4ff88dd31f274cc7b1b89a0cb1afb0dd/users/aa2b34d63f6649c3ac287f791be3adf8/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:57:25.056941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d837309-ddca-406e-b074-a10216c24a18 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:25.057233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d837309-ddca-406e-b074-a10216c24a18 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:25.057513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d837309-ddca-406e-b074-a10216c24a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:25.085675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9d837309-ddca-406e-b074-a10216c24a18 None None] MFA Rules not processed for user `aa2b34d63f6649c3ac287f791be3adf8`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:57:25.086890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9d837309-ddca-406e-b074-a10216c24a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:25.156485 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1237/2493] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:57:25 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:57:25.167805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82949a0-7ed3-4b76-9794-80e203fd44db admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:25.169617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b82949a0-7ed3-4b76-9794-80e203fd44db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:25.235614 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82949a0-7ed3-4b76-9794-80e203fd44db admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:25.239578 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4d699-bd7f-4991-8463-0e0991b4e969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:25.240255 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4d699-bd7f-4991-8463-0e0991b4e969 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:25.240540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4d699-bd7f-4991-8463-0e0991b4e969 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:25.240816 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4d699-bd7f-4991-8463-0e0991b4e969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:25.249826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4d699-bd7f-4991-8463-0e0991b4e969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:25.251386 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4d699-bd7f-4991-8463-0e0991b4e969 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:25.260689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1257/2494] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:57:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:57:25.335030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d837309-ddca-406e-b074-a10216c24a18 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:25.345760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d837309-ddca-406e-b074-a10216c24a18 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:25.349855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0407c2-6321-410a-b86b-1a49ab473ae2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:25.350507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0407c2-6321-410a-b86b-1a49ab473ae2 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:25.350790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0407c2-6321-410a-b86b-1a49ab473ae2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:25.351065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0407c2-6321-410a-b86b-1a49ab473ae2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:25.411190 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0407c2-6321-410a-b86b-1a49ab473ae2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:25.412816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0407c2-6321-410a-b86b-1a49ab473ae2 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:25.422909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1238/2495] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:57:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:57:35.256541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:57:35.418057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=67 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:57:35.742301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4d699-bd7f-4991-8463-0e0991b4e969 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:57:35.743782 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f4b4d699-bd7f-4991-8463-0e0991b4e969 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:35.822389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4d699-bd7f-4991-8463-0e0991b4e969 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:57:35.828640 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b0c30-7c51-4850-878c-d170d3a5c577 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:57:35.829368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b0c30-7c51-4850-878c-d170d3a5c577 service placement] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:35.829653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b0c30-7c51-4850-878c-d170d3a5c577 service placement] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:35.829931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b0c30-7c51-4850-878c-d170d3a5c577 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:35.832044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d97b0c30-7c51-4850-878c-d170d3a5c577 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:35.844144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b0c30-7c51-4850-878c-d170d3a5c577 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:57:35.845847 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b0c30-7c51-4850-878c-d170d3a5c577 service placement] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:57:35.856378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1258/2496] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:57:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:57:39.378674 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f5381-9c75-4643-80e5-19b298306135 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:57:39.379195 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f5381-9c75-4643-80e5-19b298306135 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:57:39.379305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f5381-9c75-4643-80e5-19b298306135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:57:39.402231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-bf8f5381-9c75-4643-80e5-19b298306135 None None] MFA Rules not processed for user `b10d2c10313c4be2a8c11cff1eed8852`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:57:39.403713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bf8f5381-9c75-4643-80e5-19b298306135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:57:39.469215 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1239/2497] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:57:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:57:45.851715 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=32 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:57:49.462307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=25 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:04.361610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f9746-2f6d-41ad-831c-79399672d9ba None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:04.362279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f9746-2f6d-41ad-831c-79399672d9ba None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:04.362279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f9746-2f6d-41ad-831c-79399672d9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:04.390176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-778f9746-2f6d-41ad-831c-79399672d9ba None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:04.391426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-778f9746-2f6d-41ad-831c-79399672d9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:04.456832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1259/2498] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:58:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:04.500339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f5381-9c75-4643-80e5-19b298306135 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:04.501848 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-bf8f5381-9c75-4643-80e5-19b298306135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:04.572943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f5381-9c75-4643-80e5-19b298306135 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:04.579175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1aba1-f34a-4a71-ade9-d14c2b300a42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:04.579831 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1aba1-f34a-4a71-ade9-d14c2b300a42 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:04.580144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1aba1-f34a-4a71-ade9-d14c2b300a42 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:04.580407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1aba1-f34a-4a71-ade9-d14c2b300a42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:04.589066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1aba1-f34a-4a71-ade9-d14c2b300a42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:04.590665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1aba1-f34a-4a71-ade9-d14c2b300a42 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:04.599030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1240/2499] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:58:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:58:12.765965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f9746-2f6d-41ad-831c-79399672d9ba None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:12.838626 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f9746-2f6d-41ad-831c-79399672d9ba None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:12.843761 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:12.844937 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:12.845299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:12.846389 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] PATH_INFO: `/v3/users/dd570726eb504cb4b9382b0237bd04d1` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:12.849116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd570726eb504cb4b9382b0237bd04d1)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:12.853064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:12.914098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd570726eb504cb4b9382b0237bd04d1'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:12.915669 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd570726eb504cb4b9382b0237bd04d1'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:12.927499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd570726eb504cb4b9382b0237bd04d1'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:12.936224 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd570726eb504cb4b9382b0237bd04d1'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:12.950043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd570726eb504cb4b9382b0237bd04d1'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:12.968635 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1260/2500] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:58:12 2022] DELETE /identity/v3/users/dd570726eb504cb4b9382b0237bd04d1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:12.973150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17f07-2093-4259-8115-53b6edf25d14 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:12.973666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17f07-2093-4259-8115-53b6edf25d14 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:12.973771 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17f07-2093-4259-8115-53b6edf25d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:12.991971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e8d17f07-2093-4259-8115-53b6edf25d14 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:12.993391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e8d17f07-2093-4259-8115-53b6edf25d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:13.065343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1241/2501] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:13.075256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:13.076908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:13.091036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:13.164369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461ba56-a2a6-416b-9bc3-edd3730b3b6f admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:13.168201 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188beab-8f1b-4d9e-809b-37c5dee584ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:13.168928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188beab-8f1b-4d9e-809b-37c5dee584ab service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:13.169278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188beab-8f1b-4d9e-809b-37c5dee584ab service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:13.169638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188beab-8f1b-4d9e-809b-37c5dee584ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:13.179463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188beab-8f1b-4d9e-809b-37c5dee584ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:13.180952 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188beab-8f1b-4d9e-809b-37c5dee584ab service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:13.196046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1261/2502] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:58:13.501006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d17f07-2093-4259-8115-53b6edf25d14 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:13.502595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e8d17f07-2093-4259-8115-53b6edf25d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:13.515189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d17f07-2093-4259-8115-53b6edf25d14 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:13.520674 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:13.521545 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:13.521854 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:13.522112 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] PATH_INFO: `/v3/projects/a13cd820ac8a4dcfb8cdcb19d967accf` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:13.523422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a13cd820ac8a4dcfb8cdcb19d967accf)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:13.525426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:13.573859 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a13cd820ac8a4dcfb8cdcb19d967accf', 'request_id': 'req-839c744f-e2b7-4530-bfa0-f2bb9a7da708'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:13.583397 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a13cd820ac8a4dcfb8cdcb19d967accf', 'request_id': 'req-839c744f-e2b7-4530-bfa0-f2bb9a7da708'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:13.589453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a13cd820ac8a4dcfb8cdcb19d967accf', 'request_id': 'req-839c744f-e2b7-4530-bfa0-f2bb9a7da708'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:13.601385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] The token cache is being invalidate because project a13cd820ac8a4dcfb8cdcb19d967accf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:58:13.601673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:13.602325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:13.603902 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1242/2503] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:58:13 2022] DELETE /identity/v3/projects/a13cd820ac8a4dcfb8cdcb19d967accf => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:13.607959 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188beab-8f1b-4d9e-809b-37c5dee584ab service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:13.618071 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188beab-8f1b-4d9e-809b-37c5dee584ab service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:13.684145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ee942-79f5-4ad6-a4d3-948b85086e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:13.685026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ee942-79f5-4ad6-a4d3-948b85086e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:13.685311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ee942-79f5-4ad6-a4d3-948b85086e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:13.685585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ee942-79f5-4ad6-a4d3-948b85086e16 admin admin] PATH_INFO: `/v3/users/aa2b34d63f6649c3ac287f791be3adf8` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:13.686851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ee942-79f5-4ad6-a4d3-948b85086e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa2b34d63f6649c3ac287f791be3adf8)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:13.688826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ee942-79f5-4ad6-a4d3-948b85086e16 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:13.736188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-419ee942-79f5-4ad6-a4d3-948b85086e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa2b34d63f6649c3ac287f791be3adf8'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:13.736848 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-419ee942-79f5-4ad6-a4d3-948b85086e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa2b34d63f6649c3ac287f791be3adf8'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:13.744513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-419ee942-79f5-4ad6-a4d3-948b85086e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa2b34d63f6649c3ac287f791be3adf8'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:13.751658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-419ee942-79f5-4ad6-a4d3-948b85086e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa2b34d63f6649c3ac287f791be3adf8'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:13.764392 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-419ee942-79f5-4ad6-a4d3-948b85086e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa2b34d63f6649c3ac287f791be3adf8'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:13.770260 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1262/2504] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:58:13 2022] DELETE /identity/v3/users/aa2b34d63f6649c3ac287f791be3adf8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:14.046863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:14.048713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:14.062599 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c744f-e2b7-4530-bfa0-f2bb9a7da708 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:14.075583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:14.127718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db6c0f-3635-43ef-ba1c-96da78ad1c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:14.128559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db6c0f-3635-43ef-ba1c-96da78ad1c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:14.128847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db6c0f-3635-43ef-ba1c-96da78ad1c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:14.129121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db6c0f-3635-43ef-ba1c-96da78ad1c99 admin admin] PATH_INFO: `/v3/projects/4ff88dd31f274cc7b1b89a0cb1afb0dd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:14.130327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db6c0f-3635-43ef-ba1c-96da78ad1c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff88dd31f274cc7b1b89a0cb1afb0dd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:14.132315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db6c0f-3635-43ef-ba1c-96da78ad1c99 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:14.182552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-83db6c0f-3635-43ef-ba1c-96da78ad1c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff88dd31f274cc7b1b89a0cb1afb0dd', 'request_id': 'req-83db6c0f-3635-43ef-ba1c-96da78ad1c99'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:14.190640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-83db6c0f-3635-43ef-ba1c-96da78ad1c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff88dd31f274cc7b1b89a0cb1afb0dd', 'request_id': 'req-83db6c0f-3635-43ef-ba1c-96da78ad1c99'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:14.196551 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-83db6c0f-3635-43ef-ba1c-96da78ad1c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff88dd31f274cc7b1b89a0cb1afb0dd', 'request_id': 'req-83db6c0f-3635-43ef-ba1c-96da78ad1c99'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:14.208994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-83db6c0f-3635-43ef-ba1c-96da78ad1c99 admin admin] The token cache is being invalidate because project 4ff88dd31f274cc7b1b89a0cb1afb0dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:58:14.209283 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-83db6c0f-3635-43ef-ba1c-96da78ad1c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:14.209987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-83db6c0f-3635-43ef-ba1c-96da78ad1c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:14.211596 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1243/2505] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:58:14 2022] DELETE /identity/v3/projects/4ff88dd31f274cc7b1b89a0cb1afb0dd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:14.216773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4978f-e424-4889-b505-f6574a55ff5d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:14.217120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4978f-e424-4889-b505-f6574a55ff5d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:14.217484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4978f-e424-4889-b505-f6574a55ff5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:14.237213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d2f4978f-e424-4889-b505-f6574a55ff5d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:14.238960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d2f4978f-e424-4889-b505-f6574a55ff5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:14.245829 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:14.307093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1263/2506] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:14.315228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6502c35-f52d-4b98-9a2f-5adcf7ab6cbf None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:14.315520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6502c35-f52d-4b98-9a2f-5adcf7ab6cbf None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:14.315801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6502c35-f52d-4b98-9a2f-5adcf7ab6cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:14.334484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e6502c35-f52d-4b98-9a2f-5adcf7ab6cbf None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:14.335685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e6502c35-f52d-4b98-9a2f-5adcf7ab6cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:14.345231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1244/2507] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:14.349579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4978f-e424-4889-b505-f6574a55ff5d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:14.360383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4978f-e424-4889-b505-f6574a55ff5d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:14.365333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846ab77-6f35-4c0b-943f-f7b8732d1b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:14.366018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846ab77-6f35-4c0b-943f-f7b8732d1b05 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:14.366296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846ab77-6f35-4c0b-943f-f7b8732d1b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:14.366572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846ab77-6f35-4c0b-943f-f7b8732d1b05 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:14.367295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846ab77-6f35-4c0b-943f-f7b8732d1b05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:58:14.367605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846ab77-6f35-4c0b-943f-f7b8732d1b05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:14.369288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846ab77-6f35-4c0b-943f-f7b8732d1b05 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:14.381800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1264/2508] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:58:14.386190 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6502c35-f52d-4b98-9a2f-5adcf7ab6cbf None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:14.395963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6502c35-f52d-4b98-9a2f-5adcf7ab6cbf None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:14.402366 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a1136-8084-41f1-b67b-87bb231fea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:14.403036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a1136-8084-41f1-b67b-87bb231fea45 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:14.403323 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a1136-8084-41f1-b67b-87bb231fea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:14.403638 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a1136-8084-41f1-b67b-87bb231fea45 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:14.404375 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a1136-8084-41f1-b67b-87bb231fea45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:14.406381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a1136-8084-41f1-b67b-87bb231fea45 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:14.437197 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1245/2509] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:58:14 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:14.438504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846ab77-6f35-4c0b-943f-f7b8732d1b05 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:14.448510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846ab77-6f35-4c0b-943f-f7b8732d1b05 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:14.513243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58931c-80c1-4ecb-9a9f-8c51a918b618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:14.513243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58931c-80c1-4ecb-9a9f-8c51a918b618 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:14.513629 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58931c-80c1-4ecb-9a9f-8c51a918b618 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:14.513723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58931c-80c1-4ecb-9a9f-8c51a918b618 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:14.514414 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58931c-80c1-4ecb-9a9f-8c51a918b618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:14.516543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58931c-80c1-4ecb-9a9f-8c51a918b618 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:14.543576 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1265/2510] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:58:14 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:14.544982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a1136-8084-41f1-b67b-87bb231fea45 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:14.556955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a1136-8084-41f1-b67b-87bb231fea45 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:14.562971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a702cee-df7d-4ddf-b37a-8542bfe4dd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:14.563736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a702cee-df7d-4ddf-b37a-8542bfe4dd4f admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:14.564207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a702cee-df7d-4ddf-b37a-8542bfe4dd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:14.564525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a702cee-df7d-4ddf-b37a-8542bfe4dd4f admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:14.565366 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a702cee-df7d-4ddf-b37a-8542bfe4dd4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:14.567179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a702cee-df7d-4ddf-b37a-8542bfe4dd4f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:14.582527 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1246/2511] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:14.583880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58931c-80c1-4ecb-9a9f-8c51a918b618 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:14.600786 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58931c-80c1-4ecb-9a9f-8c51a918b618 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:14.605484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1af5d8-a9aa-44ef-971a-8ef4cce84ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:14.606156 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1af5d8-a9aa-44ef-971a-8ef4cce84ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:14.606431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1af5d8-a9aa-44ef-971a-8ef4cce84ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:14.606704 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1af5d8-a9aa-44ef-971a-8ef4cce84ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:14.607547 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1af5d8-a9aa-44ef-971a-8ef4cce84ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:14.609045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1af5d8-a9aa-44ef-971a-8ef4cce84ed1 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:14.619436 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1266/2512] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:14.623631 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a702cee-df7d-4ddf-b37a-8542bfe4dd4f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:14.634401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a702cee-df7d-4ddf-b37a-8542bfe4dd4f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:14.638913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591dc35-ebdd-479e-a343-31fbfa727af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:14.639845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591dc35-ebdd-479e-a343-31fbfa727af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:14.640191 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591dc35-ebdd-479e-a343-31fbfa727af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:14.640473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591dc35-ebdd-479e-a343-31fbfa727af5 admin admin] PATH_INFO: `/v3/projects/08945f5a1563432fb9ab4545fc714fdd/users/f9146e84824b4f1985fa939f96a2b9bc/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:14.657689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591dc35-ebdd-479e-a343-31fbfa727af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08945f5a1563432fb9ab4545fc714fdd, user_id=f9146e84824b4f1985fa939f96a2b9bc, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:14.660631 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591dc35-ebdd-479e-a343-31fbfa727af5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:14.673946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1247/2513] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:58:14 2022] PUT /identity/v3/projects/08945f5a1563432fb9ab4545fc714fdd/users/f9146e84824b4f1985fa939f96a2b9bc/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:14.684918 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81babe9-de2a-400b-94cc-61e1c4639dd1 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:14.685212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81babe9-de2a-400b-94cc-61e1c4639dd1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:14.685491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81babe9-de2a-400b-94cc-61e1c4639dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:14.723681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-d81babe9-de2a-400b-94cc-61e1c4639dd1 None None] MFA Rules not processed for user `f9146e84824b4f1985fa939f96a2b9bc`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:14.724954 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d81babe9-de2a-400b-94cc-61e1c4639dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:14.789360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1267/2514] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:14.793993 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591dc35-ebdd-479e-a343-31fbfa727af5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:14.837217 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:14.859015 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591dc35-ebdd-479e-a343-31fbfa727af5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:14.863981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52727cb3-2e57-4155-a3ba-dd5cf1894592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:14.864696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52727cb3-2e57-4155-a3ba-dd5cf1894592 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:14.864966 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52727cb3-2e57-4155-a3ba-dd5cf1894592 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:14.865342 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52727cb3-2e57-4155-a3ba-dd5cf1894592 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:14.866042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52727cb3-2e57-4155-a3ba-dd5cf1894592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:14.868201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52727cb3-2e57-4155-a3ba-dd5cf1894592 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:14.899821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1248/2515] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:58:14 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:14.901377 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81babe9-de2a-400b-94cc-61e1c4639dd1 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:14.912759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81babe9-de2a-400b-94cc-61e1c4639dd1 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:14.916757 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:14.982821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a3cfc-4d62-473b-99b5-b7d240b598ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:14.983503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a3cfc-4d62-473b-99b5-b7d240b598ac admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:14.983789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a3cfc-4d62-473b-99b5-b7d240b598ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:14.984111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a3cfc-4d62-473b-99b5-b7d240b598ac admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:14.984805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a3cfc-4d62-473b-99b5-b7d240b598ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:14.986763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a3cfc-4d62-473b-99b5-b7d240b598ac admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:15.007638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1268/2516] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:58:14 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:15.011719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52727cb3-2e57-4155-a3ba-dd5cf1894592 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:15.022370 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52727cb3-2e57-4155-a3ba-dd5cf1894592 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:15.027564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87b2c9-ae4e-4542-b0c9-5d88823fa2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:15.028268 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87b2c9-ae4e-4542-b0c9-5d88823fa2fa admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:15.028552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87b2c9-ae4e-4542-b0c9-5d88823fa2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:15.028832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87b2c9-ae4e-4542-b0c9-5d88823fa2fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:15.029566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87b2c9-ae4e-4542-b0c9-5d88823fa2fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:15.031185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87b2c9-ae4e-4542-b0c9-5d88823fa2fa admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:15.041624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1249/2517] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:15.045559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a3cfc-4d62-473b-99b5-b7d240b598ac admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:15.056034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a3cfc-4d62-473b-99b5-b7d240b598ac admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:15.061577 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69278c11-b8a4-4c56-9046-10ef11c03dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:15.062767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69278c11-b8a4-4c56-9046-10ef11c03dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:15.063142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69278c11-b8a4-4c56-9046-10ef11c03dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:15.063510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69278c11-b8a4-4c56-9046-10ef11c03dc9 admin admin] PATH_INFO: `/v3/projects/94b1848cc64c477d95f598edb0e53c94/users/6fb0fc090ecc4b5497a91ad9fad90eda/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:15.079969 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69278c11-b8a4-4c56-9046-10ef11c03dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b1848cc64c477d95f598edb0e53c94, user_id=6fb0fc090ecc4b5497a91ad9fad90eda, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:15.082838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69278c11-b8a4-4c56-9046-10ef11c03dc9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:15.096775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1269/2518] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:58:15 2022] PUT /identity/v3/projects/94b1848cc64c477d95f598edb0e53c94/users/6fb0fc090ecc4b5497a91ad9fad90eda/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:15.100756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87b2c9-ae4e-4542-b0c9-5d88823fa2fa admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:15.110691 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87b2c9-ae4e-4542-b0c9-5d88823fa2fa admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:15.176287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b206a4e-fcc4-4a4e-bbff-dff45c848f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:15.176885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b206a4e-fcc4-4a4e-bbff-dff45c848f62 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:15.177167 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b206a4e-fcc4-4a4e-bbff-dff45c848f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:15.177434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b206a4e-fcc4-4a4e-bbff-dff45c848f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:15.178176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b206a4e-fcc4-4a4e-bbff-dff45c848f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:15.179785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b206a4e-fcc4-4a4e-bbff-dff45c848f62 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:15.190143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1250/2519] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:15.194587 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69278c11-b8a4-4c56-9046-10ef11c03dc9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:15.205347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69278c11-b8a4-4c56-9046-10ef11c03dc9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:15.210610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ef568-1366-4697-92d9-375c31bf3c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:15.211514 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ef568-1366-4697-92d9-375c31bf3c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:15.211810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ef568-1366-4697-92d9-375c31bf3c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:15.212232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ef568-1366-4697-92d9-375c31bf3c8b admin admin] PATH_INFO: `/v3/projects/94b1848cc64c477d95f598edb0e53c94/users/6fb0fc090ecc4b5497a91ad9fad90eda/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:15.214222 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ef568-1366-4697-92d9-375c31bf3c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b1848cc64c477d95f598edb0e53c94, user_id=6fb0fc090ecc4b5497a91ad9fad90eda, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:15.216940 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ef568-1366-4697-92d9-375c31bf3c8b admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:15.228572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1270/2520] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:58:15 2022] PUT /identity/v3/projects/94b1848cc64c477d95f598edb0e53c94/users/6fb0fc090ecc4b5497a91ad9fad90eda/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:15.239994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6ddb0-5d67-44ad-9451-6bbb8a4df25f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:15.240305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6ddb0-5d67-44ad-9451-6bbb8a4df25f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:15.240586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6ddb0-5d67-44ad-9451-6bbb8a4df25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:15.267140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-91a6ddb0-5d67-44ad-9451-6bbb8a4df25f None None] MFA Rules not processed for user `6fb0fc090ecc4b5497a91ad9fad90eda`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:15.268416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-91a6ddb0-5d67-44ad-9451-6bbb8a4df25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:15.340225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1251/2521] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:15 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:15.351483 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ef568-1366-4697-92d9-375c31bf3c8b admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:15.353401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-896ef568-1366-4697-92d9-375c31bf3c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:15.443784 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ef568-1366-4697-92d9-375c31bf3c8b admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:15.449424 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72e312-82b2-4bd8-94c5-753fe8a749c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:15.450079 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72e312-82b2-4bd8-94c5-753fe8a749c8 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:15.450365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72e312-82b2-4bd8-94c5-753fe8a749c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:15.450644 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72e312-82b2-4bd8-94c5-753fe8a749c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:15.461114 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72e312-82b2-4bd8-94c5-753fe8a749c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:15.462583 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72e312-82b2-4bd8-94c5-753fe8a749c8 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:15.472889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1271/2522] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:15 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:16.884106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6ddb0-5d67-44ad-9451-6bbb8a4df25f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:16.924437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:16.952417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6ddb0-5d67-44ad-9451-6bbb8a4df25f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:16.960694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:16.961610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:16.961889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:16.962154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] PATH_INFO: `/v3/users/f9146e84824b4f1985fa939f96a2b9bc` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:16.964295 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9146e84824b4f1985fa939f96a2b9bc)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:16.966395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:17.015347 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9146e84824b4f1985fa939f96a2b9bc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.023046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9146e84824b4f1985fa939f96a2b9bc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.023702 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9146e84824b4f1985fa939f96a2b9bc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.030978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9146e84824b4f1985fa939f96a2b9bc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.042536 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9146e84824b4f1985fa939f96a2b9bc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.048430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1252/2523] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:58:16 2022] DELETE /identity/v3/users/f9146e84824b4f1985fa939f96a2b9bc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:58:17.053164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ee8d9-e7a9-437a-a380-2a70d32bcbf2 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:17.053660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ee8d9-e7a9-437a-a380-2a70d32bcbf2 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:17.053789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ee8d9-e7a9-437a-a380-2a70d32bcbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:17.072569 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-e55ee8d9-e7a9-437a-a380-2a70d32bcbf2 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:17.073765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e55ee8d9-e7a9-437a-a380-2a70d32bcbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:17.114047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:17.135988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1272/2524] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:17.146120 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:17.147669 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:17.209704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074af5ed-e5db-4b4c-8c60-d837e23c9b05 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:17.214183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8baf8-91e9-4a06-8737-30a3853cb744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:17.214849 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8baf8-91e9-4a06-8737-30a3853cb744 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:17.215136 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8baf8-91e9-4a06-8737-30a3853cb744 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:17.215416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8baf8-91e9-4a06-8737-30a3853cb744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:17.224869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8baf8-91e9-4a06-8737-30a3853cb744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:17.226267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8baf8-91e9-4a06-8737-30a3853cb744 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:17.236315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1253/2525] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:17.513205 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ee8d9-e7a9-437a-a380-2a70d32bcbf2 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:17.515149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e55ee8d9-e7a9-437a-a380-2a70d32bcbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:17.528330 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ee8d9-e7a9-437a-a380-2a70d32bcbf2 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:17.534603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:17.536119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:17.536472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:17.536815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] PATH_INFO: `/v3/projects/08945f5a1563432fb9ab4545fc714fdd` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:17.538102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08945f5a1563432fb9ab4545fc714fdd)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:17.540535 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:17.590647 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08945f5a1563432fb9ab4545fc714fdd', 'request_id': 'req-bf247529-ec5a-4475-b071-8b0af2154bc2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.598178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08945f5a1563432fb9ab4545fc714fdd', 'request_id': 'req-bf247529-ec5a-4475-b071-8b0af2154bc2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.604593 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08945f5a1563432fb9ab4545fc714fdd', 'request_id': 'req-bf247529-ec5a-4475-b071-8b0af2154bc2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.614778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] The token cache is being invalidate because project 08945f5a1563432fb9ab4545fc714fdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:58:17.615059 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.615684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.617230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1273/2526] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:58:17 2022] DELETE /identity/v3/projects/08945f5a1563432fb9ab4545fc714fdd => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:58:17.621383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8baf8-91e9-4a06-8737-30a3853cb744 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:17.631215 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8baf8-91e9-4a06-8737-30a3853cb744 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:17.682547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:17.689694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7844f-acb7-46ad-95be-ca0cf9272997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:17.690586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7844f-acb7-46ad-95be-ca0cf9272997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:17.691235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7844f-acb7-46ad-95be-ca0cf9272997 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:17.691616 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7844f-acb7-46ad-95be-ca0cf9272997 admin admin] PATH_INFO: `/v3/users/6fb0fc090ecc4b5497a91ad9fad90eda` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:17.692975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7844f-acb7-46ad-95be-ca0cf9272997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fb0fc090ecc4b5497a91ad9fad90eda)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:17.695431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7844f-acb7-46ad-95be-ca0cf9272997 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:17.752798 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fef7844f-acb7-46ad-95be-ca0cf9272997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fb0fc090ecc4b5497a91ad9fad90eda'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.760903 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fef7844f-acb7-46ad-95be-ca0cf9272997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fb0fc090ecc4b5497a91ad9fad90eda'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.761458 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fef7844f-acb7-46ad-95be-ca0cf9272997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fb0fc090ecc4b5497a91ad9fad90eda'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.768611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fef7844f-acb7-46ad-95be-ca0cf9272997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fb0fc090ecc4b5497a91ad9fad90eda'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.780825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-fef7844f-acb7-46ad-95be-ca0cf9272997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fb0fc090ecc4b5497a91ad9fad90eda'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:17.786832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1254/2527] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:58:17 2022] DELETE /identity/v3/users/6fb0fc090ecc4b5497a91ad9fad90eda => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:18.091524 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:18.093107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:18.106316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf247529-ec5a-4475-b071-8b0af2154bc2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:18.176426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:18.177306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:18.177589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:18.177859 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2 admin admin] PATH_INFO: `/v3/projects/94b1848cc64c477d95f598edb0e53c94` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:18.179066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94b1848cc64c477d95f598edb0e53c94)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:18.181126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:18.230245 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94b1848cc64c477d95f598edb0e53c94', 'request_id': 'req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:18.238951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94b1848cc64c477d95f598edb0e53c94', 'request_id': 'req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:18.245869 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94b1848cc64c477d95f598edb0e53c94', 'request_id': 'req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:18.250788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:18.259163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2 admin admin] The token cache is being invalidate because project 94b1848cc64c477d95f598edb0e53c94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:58:18.259506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:18.260506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-55a80c22-a926-4dd4-adf0-01cd6b9f64a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:18.265611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1274/2528] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:58:18 2022] DELETE /identity/v3/projects/94b1848cc64c477d95f598edb0e53c94 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:18.267319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fad86-3468-4edb-b4f5-73379d39a116 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:18.267838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fad86-3468-4edb-b4f5-73379d39a116 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:18.267957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fad86-3468-4edb-b4f5-73379d39a116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:18.287775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-251fad86-3468-4edb-b4f5-73379d39a116 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:18.289006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-251fad86-3468-4edb-b4f5-73379d39a116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:18.353007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1255/2529] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:58:18.361251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af91be2-8816-46c1-8681-48589f9b99d6 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:18.361649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af91be2-8816-46c1-8681-48589f9b99d6 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:18.362048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af91be2-8816-46c1-8681-48589f9b99d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:18.382668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-0af91be2-8816-46c1-8681-48589f9b99d6 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:18.384051 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0af91be2-8816-46c1-8681-48589f9b99d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:18.395296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1275/2530] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:18.400380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fad86-3468-4edb-b4f5-73379d39a116 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:18.411728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fad86-3468-4edb-b4f5-73379d39a116 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:18.417499 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc87e20-d1f7-4f74-bd57-fc01aaa7228b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:18.418178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc87e20-d1f7-4f74-bd57-fc01aaa7228b admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:18.418458 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc87e20-d1f7-4f74-bd57-fc01aaa7228b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:18.418732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc87e20-d1f7-4f74-bd57-fc01aaa7228b admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:18.419604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc87e20-d1f7-4f74-bd57-fc01aaa7228b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:58:18.419910 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc87e20-d1f7-4f74-bd57-fc01aaa7228b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:18.422189 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc87e20-d1f7-4f74-bd57-fc01aaa7228b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:18.436498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1256/2531] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:58:18.440686 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af91be2-8816-46c1-8681-48589f9b99d6 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:18.452508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af91be2-8816-46c1-8681-48589f9b99d6 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:18.458664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38310c49-b5c8-4f96-a543-59299462d12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:18.459699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38310c49-b5c8-4f96-a543-59299462d12a admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:18.460391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38310c49-b5c8-4f96-a543-59299462d12a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:18.460742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38310c49-b5c8-4f96-a543-59299462d12a admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:18.461634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38310c49-b5c8-4f96-a543-59299462d12a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:18.463706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38310c49-b5c8-4f96-a543-59299462d12a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:18.520394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1276/2532] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:58:18 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:58:18.521552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc87e20-d1f7-4f74-bd57-fc01aaa7228b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:18.533153 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc87e20-d1f7-4f74-bd57-fc01aaa7228b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:18.593983 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d17450-1377-446b-9e9f-86f481f550b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:18.594667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d17450-1377-446b-9e9f-86f481f550b3 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:18.594955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d17450-1377-446b-9e9f-86f481f550b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:18.595238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d17450-1377-446b-9e9f-86f481f550b3 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:18.595920 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d17450-1377-446b-9e9f-86f481f550b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:18.597981 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d17450-1377-446b-9e9f-86f481f550b3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:18.606669 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:18.624088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1257/2533] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:58:18 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:18.628539 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38310c49-b5c8-4f96-a543-59299462d12a admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:18.638849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38310c49-b5c8-4f96-a543-59299462d12a admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:18.643506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed0e07-cc3a-4c20-9b4d-4977f053c589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:18.644378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed0e07-cc3a-4c20-9b4d-4977f053c589 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:18.644659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed0e07-cc3a-4c20-9b4d-4977f053c589 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:18.644926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed0e07-cc3a-4c20-9b4d-4977f053c589 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:18.645677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed0e07-cc3a-4c20-9b4d-4977f053c589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:18.647272 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed0e07-cc3a-4c20-9b4d-4977f053c589 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:18.658248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1277/2534] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:18.662516 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d17450-1377-446b-9e9f-86f481f550b3 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:18.672931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d17450-1377-446b-9e9f-86f481f550b3 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:18.677718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a3b86-d8c1-4737-9ea2-45113b04d536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:18.678302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a3b86-d8c1-4737-9ea2-45113b04d536 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:18.678581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a3b86-d8c1-4737-9ea2-45113b04d536 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:18.678857 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a3b86-d8c1-4737-9ea2-45113b04d536 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:18.679568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a3b86-d8c1-4737-9ea2-45113b04d536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:18.681215 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a3b86-d8c1-4737-9ea2-45113b04d536 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:18.692038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1258/2535] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:18.696146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed0e07-cc3a-4c20-9b4d-4977f053c589 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:18.706238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed0e07-cc3a-4c20-9b4d-4977f053c589 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:18.711157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74803105-077a-4e21-8e7b-fd156f67b2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:18.712072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74803105-077a-4e21-8e7b-fd156f67b2c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:18.712354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74803105-077a-4e21-8e7b-fd156f67b2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:18.712627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74803105-077a-4e21-8e7b-fd156f67b2c8 admin admin] PATH_INFO: `/v3/projects/534d33178c47478fa57289cdcf2e9448/users/66f0c40e34fb4e67b24dbf3d1ff5879b/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:18.727387 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74803105-077a-4e21-8e7b-fd156f67b2c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=534d33178c47478fa57289cdcf2e9448, user_id=66f0c40e34fb4e67b24dbf3d1ff5879b, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:18.730064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74803105-077a-4e21-8e7b-fd156f67b2c8 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:18.743269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1278/2536] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:58:18 2022] PUT /identity/v3/projects/534d33178c47478fa57289cdcf2e9448/users/66f0c40e34fb4e67b24dbf3d1ff5879b/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:18.754427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580f715-2798-4a81-b51f-50aa3cbed0dc None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:18.754767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580f715-2798-4a81-b51f-50aa3cbed0dc None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:18.755044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580f715-2798-4a81-b51f-50aa3cbed0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:18.782934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-5580f715-2798-4a81-b51f-50aa3cbed0dc None None] MFA Rules not processed for user `66f0c40e34fb4e67b24dbf3d1ff5879b`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:18.784159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5580f715-2798-4a81-b51f-50aa3cbed0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:18.848224 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1259/2537] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:18.852976 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74803105-077a-4e21-8e7b-fd156f67b2c8 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:18.919418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74803105-077a-4e21-8e7b-fd156f67b2c8 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:18.924111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef96fc4-cbe2-4268-8356-3d3dcd8ff5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:18.924758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef96fc4-cbe2-4268-8356-3d3dcd8ff5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:18.925038 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef96fc4-cbe2-4268-8356-3d3dcd8ff5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:18.925332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef96fc4-cbe2-4268-8356-3d3dcd8ff5a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:18.926061 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef96fc4-cbe2-4268-8356-3d3dcd8ff5a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:18.927966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef96fc4-cbe2-4268-8356-3d3dcd8ff5a8 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:18.959709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1279/2538] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:58:18 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:58:18.961676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580f715-2798-4a81-b51f-50aa3cbed0dc None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:18.973321 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580f715-2798-4a81-b51f-50aa3cbed0dc None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.030070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314aabe-bc17-4845-b23d-d002bfa73b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.030769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314aabe-bc17-4845-b23d-d002bfa73b98 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.031047 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314aabe-bc17-4845-b23d-d002bfa73b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.031317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314aabe-bc17-4845-b23d-d002bfa73b98 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.031997 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314aabe-bc17-4845-b23d-d002bfa73b98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.033969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314aabe-bc17-4845-b23d-d002bfa73b98 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.057737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1260/2539] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:58:18 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:19.061780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef96fc4-cbe2-4268-8356-3d3dcd8ff5a8 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.073799 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef96fc4-cbe2-4268-8356-3d3dcd8ff5a8 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.079854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b90eae-c93e-4961-9f9f-f3f1fcc9862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.080572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b90eae-c93e-4961-9f9f-f3f1fcc9862b admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.080859 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b90eae-c93e-4961-9f9f-f3f1fcc9862b admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.081146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b90eae-c93e-4961-9f9f-f3f1fcc9862b admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.081916 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b90eae-c93e-4961-9f9f-f3f1fcc9862b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.083603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b90eae-c93e-4961-9f9f-f3f1fcc9862b admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.095103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1280/2540] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:19.098119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d7b96-3b85-46bf-897a-72a7861378c7 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.098562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d7b96-3b85-46bf-897a-72a7861378c7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.098762 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d7b96-3b85-46bf-897a-72a7861378c7 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.099417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b90eae-c93e-4961-9f9f-f3f1fcc9862b admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.100618 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1261/2541] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:58:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:58:19.106250 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ff9d8-fc90-4635-9a9f-922a2c752919 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.106543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ff9d8-fc90-4635-9a9f-922a2c752919 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.106834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ff9d8-fc90-4635-9a9f-922a2c752919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.111497 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b90eae-c93e-4961-9f9f-f3f1fcc9862b admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.116958 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a66b5-4506-461a-ae32-765e2e08a5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.117886 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a66b5-4506-461a-ae32-765e2e08a5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.118571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a66b5-4506-461a-ae32-765e2e08a5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.119130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a66b5-4506-461a-ae32-765e2e08a5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.120111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a66b5-4506-461a-ae32-765e2e08a5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.121906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a66b5-4506-461a-ae32-765e2e08a5a2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.133426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1281/2542] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:19.140089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a66b5-4506-461a-ae32-765e2e08a5a2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.141031 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-e03ff9d8-fc90-4635-9a9f-922a2c752919 None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:19.142486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e03ff9d8-fc90-4635-9a9f-922a2c752919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:19.151983 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a66b5-4506-461a-ae32-765e2e08a5a2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.157935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c3f22-490e-4aeb-a357-b1735aa54a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.158471 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c3f22-490e-4aeb-a357-b1735aa54a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.158749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c3f22-490e-4aeb-a357-b1735aa54a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.159024 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c3f22-490e-4aeb-a357-b1735aa54a8e admin admin] PATH_INFO: `/v3/projects/9e735bc57e7140b3852f6db42c612137/users/7888af5f06164a5eb1d874ac8ec42d84/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.175973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c3f22-490e-4aeb-a357-b1735aa54a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e735bc57e7140b3852f6db42c612137, user_id=7888af5f06164a5eb1d874ac8ec42d84, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.178927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c3f22-490e-4aeb-a357-b1735aa54a8e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.192814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:19.195980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1282/2543] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:58:19 2022] PUT /identity/v3/projects/9e735bc57e7140b3852f6db42c612137/users/7888af5f06164a5eb1d874ac8ec42d84/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:19.208479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147414b-6c89-45a8-b9bb-abfc6b5ef35d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.209042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147414b-6c89-45a8-b9bb-abfc6b5ef35d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.209491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147414b-6c89-45a8-b9bb-abfc6b5ef35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.233564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1262/2544] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:58:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:19.241603 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-2147414b-6c89-45a8-b9bb-abfc6b5ef35d None None] MFA Rules not processed for user `7888af5f06164a5eb1d874ac8ec42d84`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:19.242798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-2147414b-6c89-45a8-b9bb-abfc6b5ef35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:19.245102 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ff9d8-fc90-4635-9a9f-922a2c752919 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.246463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-e03ff9d8-fc90-4635-9a9f-922a2c752919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:19.266827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:19.303118 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ff9d8-fc90-4635-9a9f-922a2c752919 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.317830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1283/2545] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:19.318925 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147414b-6c89-45a8-b9bb-abfc6b5ef35d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.363562 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba7f1c-57e7-4c77-a75c-779779052cdf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7297224dc73c4358b60f1b096c0fb5bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.364261 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba7f1c-57e7-4c77-a75c-779779052cdf service glance] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.364535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba7f1c-57e7-4c77-a75c-779779052cdf service glance] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.364796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba7f1c-57e7-4c77-a75c-779779052cdf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.384171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147414b-6c89-45a8-b9bb-abfc6b5ef35d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.388876 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9848f-2ab3-4b95-b27f-2ca49b1ea728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.389540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9848f-2ab3-4b95-b27f-2ca49b1ea728 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.389826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9848f-2ab3-4b95-b27f-2ca49b1ea728 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.390106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9848f-2ab3-4b95-b27f-2ca49b1ea728 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.390798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9848f-2ab3-4b95-b27f-2ca49b1ea728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.392758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9848f-2ab3-4b95-b27f-2ca49b1ea728 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.421660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1284/2546] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:58:19 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:58:19.425781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9848f-2ab3-4b95-b27f-2ca49b1ea728 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.447614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba7f1c-57e7-4c77-a75c-779779052cdf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.449118 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba7f1c-57e7-4c77-a75c-779779052cdf service glance] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.461991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1263/2547] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:58:19.489895 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9848f-2ab3-4b95-b27f-2ca49b1ea728 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.494779 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69758f4b-d153-4123-8c9f-6deeac179804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.495468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69758f4b-d153-4123-8c9f-6deeac179804 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.495753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69758f4b-d153-4123-8c9f-6deeac179804 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.496025 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69758f4b-d153-4123-8c9f-6deeac179804 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.496808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69758f4b-d153-4123-8c9f-6deeac179804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.498747 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69758f4b-d153-4123-8c9f-6deeac179804 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.521417 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1285/2548] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:58:19 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:58:19.525756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba7f1c-57e7-4c77-a75c-779779052cdf service glance] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.531742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b94bc-e9ef-4a05-9fcb-ec76eb2b62d5 None None] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.532090 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b94bc-e9ef-4a05-9fcb-ec76eb2b62d5 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.532371 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b94bc-e9ef-4a05-9fcb-ec76eb2b62d5 None None] PATH_INFO: `/` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.533746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1286/2549] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 03:58:19.537732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba7f1c-57e7-4c77-a75c-779779052cdf service glance] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.538451 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ca518-736c-4626-a1c4-043e4fb009fc None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.538708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ca518-736c-4626-a1c4-043e4fb009fc None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.539014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ca518-736c-4626-a1c4-043e4fb009fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.542453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491459e-d0f3-4294-8942-eb0bde16ca3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.543121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491459e-d0f3-4294-8942-eb0bde16ca3f admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.543397 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491459e-d0f3-4294-8942-eb0bde16ca3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.543688 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491459e-d0f3-4294-8942-eb0bde16ca3f admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.544474 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491459e-d0f3-4294-8942-eb0bde16ca3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.546081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491459e-d0f3-4294-8942-eb0bde16ca3f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.556427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1264/2550] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:19.557180 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-206ca518-736c-4626-a1c4-043e4fb009fc None None] MFA Rules not processed for user `139be21e7f644b5796d30158851ebd00`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:19.558392 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-206ca518-736c-4626-a1c4-043e4fb009fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:19.560598 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491459e-d0f3-4294-8942-eb0bde16ca3f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.570414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491459e-d0f3-4294-8942-eb0bde16ca3f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.575078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1344a-c2af-4897-9b9d-d225813b7a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.575939 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1344a-c2af-4897-9b9d-d225813b7a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.576258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1344a-c2af-4897-9b9d-d225813b7a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.576542 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d1344a-c2af-4897-9b9d-d225813b7a3c admin admin] PATH_INFO: `/v3/projects/44532b16880f4ab4a3ba47587a5204ae/users/221a3923fda849aeae9af3568793012e/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.588392 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:19.592764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d1344a-c2af-4897-9b9d-d225813b7a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44532b16880f4ab4a3ba47587a5204ae, user_id=221a3923fda849aeae9af3568793012e, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.595835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d1344a-c2af-4897-9b9d-d225813b7a3c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.600075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:19.611323 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1265/2551] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:58:19 2022] PUT /identity/v3/projects/44532b16880f4ab4a3ba47587a5204ae/users/221a3923fda849aeae9af3568793012e/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:58:19.612510 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1287/2552] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:58:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:58:19.615417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1344a-c2af-4897-9b9d-d225813b7a3c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.626534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ca518-736c-4626-a1c4-043e4fb009fc None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.627961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-206ca518-736c-4626-a1c4-043e4fb009fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:19.638168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ca518-736c-4626-a1c4-043e4fb009fc None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.679074 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d1344a-c2af-4897-9b9d-d225813b7a3c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.683835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665787fd-7983-4912-80be-f5bbad2c0ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.684543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665787fd-7983-4912-80be-f5bbad2c0ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.684828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665787fd-7983-4912-80be-f5bbad2c0ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.685108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665787fd-7983-4912-80be-f5bbad2c0ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.685839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665787fd-7983-4912-80be-f5bbad2c0ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.687481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665787fd-7983-4912-80be-f5bbad2c0ba8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.689454 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58694e12-06dc-4418-81e6-31f1ac0be9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.690098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58694e12-06dc-4418-81e6-31f1ac0be9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.690383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58694e12-06dc-4418-81e6-31f1ac0be9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.690656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58694e12-06dc-4418-81e6-31f1ac0be9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.698562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58694e12-06dc-4418-81e6-31f1ac0be9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.700047 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58694e12-06dc-4418-81e6-31f1ac0be9d3 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.705535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1266/2553] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 03:58:19.709045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1288/2554] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:19.710872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665787fd-7983-4912-80be-f5bbad2c0ba8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.722483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665787fd-7983-4912-80be-f5bbad2c0ba8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.727227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d079c-187a-460c-87ac-1121d659e03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.728178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d079c-187a-460c-87ac-1121d659e03f admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.728465 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d079c-187a-460c-87ac-1121d659e03f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.728743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d079c-187a-460c-87ac-1121d659e03f admin admin] PATH_INFO: `/v3/projects/44532b16880f4ab4a3ba47587a5204ae/users/221a3923fda849aeae9af3568793012e/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.730724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d079c-187a-460c-87ac-1121d659e03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=44532b16880f4ab4a3ba47587a5204ae, user_id=221a3923fda849aeae9af3568793012e, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.733506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d079c-187a-460c-87ac-1121d659e03f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.744301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1267/2555] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:58:19 2022] PUT /identity/v3/projects/44532b16880f4ab4a3ba47587a5204ae/users/221a3923fda849aeae9af3568793012e/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:19.755169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f293d-4784-4232-b61b-73d9f31d67d4 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.755465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f293d-4784-4232-b61b-73d9f31d67d4 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.755746 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f293d-4784-4232-b61b-73d9f31d67d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.783867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-a19f293d-4784-4232-b61b-73d9f31d67d4 None None] MFA Rules not processed for user `221a3923fda849aeae9af3568793012e`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:19.785281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a19f293d-4784-4232-b61b-73d9f31d67d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:19.793373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1957a3-49ce-4583-97df-9ac348dcb880 None None] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.793786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1957a3-49ce-4583-97df-9ac348dcb880 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.794078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1957a3-49ce-4583-97df-9ac348dcb880 None None] PATH_INFO: `/` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.795441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1268/2556] 10.222.0.20 () {54 vars in 921 bytes} [Tue Jul 12 03:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 03:58:19.801169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f60bf-deb3-45fb-bae3-652ebe63d410 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.801459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f60bf-deb3-45fb-bae3-652ebe63d410 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.801737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f60bf-deb3-45fb-bae3-652ebe63d410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.823022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4e9f60bf-deb3-45fb-bae3-652ebe63d410 None None] MFA Rules not processed for user `7138318e53ff4b7885942d49b19eb0a5`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:19.824299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4e9f60bf-deb3-45fb-bae3-652ebe63d410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:19.856214 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1289/2557] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:19.867019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f293d-4784-4232-b61b-73d9f31d67d4 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.868419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a19f293d-4784-4232-b61b-73d9f31d67d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:19.891905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1269/2558] 10.222.0.20 () {58 vars in 1018 bytes} [Tue Jul 12 03:58:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:58:19.903459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f60bf-deb3-45fb-bae3-652ebe63d410 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:19.904863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4e9f60bf-deb3-45fb-bae3-652ebe63d410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:19.925212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f293d-4784-4232-b61b-73d9f31d67d4 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.928966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1cab5-603d-4669-9eca-732cc2660c38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.929646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1cab5-603d-4669-9eca-732cc2660c38 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.929935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1cab5-603d-4669-9eca-732cc2660c38 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.930248 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1cab5-603d-4669-9eca-732cc2660c38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.941173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1cab5-603d-4669-9eca-732cc2660c38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.942641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1cab5-603d-4669-9eca-732cc2660c38 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.953748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1290/2559] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:58:19.961187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f60bf-deb3-45fb-bae3-652ebe63d410 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:19.965121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a88443-b483-4ff8-b57a-4d7e854a6b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:19.965769 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a88443-b483-4ff8-b57a-4d7e854a6b0d service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:19.966057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a88443-b483-4ff8-b57a-4d7e854a6b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:19.966334 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a88443-b483-4ff8-b57a-4d7e854a6b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:19.977374 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a88443-b483-4ff8-b57a-4d7e854a6b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:19.978817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a88443-b483-4ff8-b57a-4d7e854a6b0d service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:19.989659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1270/2560] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 03:58:20.473307 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1cab5-603d-4669-9eca-732cc2660c38 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:20.474774 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e5e1cab5-603d-4669-9eca-732cc2660c38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:20.486108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1cab5-603d-4669-9eca-732cc2660c38 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:20.489825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3d484-736f-4ac4-9301-f88a16d3ad8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:20.490466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3d484-736f-4ac4-9301-f88a16d3ad8f service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:20.490751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3d484-736f-4ac4-9301-f88a16d3ad8f service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:20.491027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3d484-736f-4ac4-9301-f88a16d3ad8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:20.503278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:20.549964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3d484-736f-4ac4-9301-f88a16d3ad8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:20.551539 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3d484-736f-4ac4-9301-f88a16d3ad8f service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:20.561110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1291/2561] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:20.719926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a88443-b483-4ff8-b57a-4d7e854a6b0d service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:20.729710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a88443-b483-4ff8-b57a-4d7e854a6b0d service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:20.734121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ccc2a-dcb9-4e4e-a151-aca8a6b2c1af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:20.734836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ccc2a-dcb9-4e4e-a151-aca8a6b2c1af service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:20.735109 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ccc2a-dcb9-4e4e-a151-aca8a6b2c1af service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:20.735384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ccc2a-dcb9-4e4e-a151-aca8a6b2c1af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:20.737270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fb1ccc2a-dcb9-4e4e-a151-aca8a6b2c1af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:20.748609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:20.816906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ccc2a-dcb9-4e4e-a151-aca8a6b2c1af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:20.818506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ccc2a-dcb9-4e4e-a151-aca8a6b2c1af service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:20.829343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1271/2562] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:21.936702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19eb2a-4599-4f40-b52a-97fb4104f173 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:21.937620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19eb2a-4599-4f40-b52a-97fb4104f173 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:21.937910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19eb2a-4599-4f40-b52a-97fb4104f173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:21.957612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-7a19eb2a-4599-4f40-b52a-97fb4104f173 None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:21.959110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7a19eb2a-4599-4f40-b52a-97fb4104f173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:22.024956 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1292/2563] 10.222.0.20 () {58 vars in 1009 bytes} [Tue Jul 12 03:58:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:22.038471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ccc2a-dcb9-4e4e-a151-aca8a6b2c1af service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:22.041076 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fb1ccc2a-dcb9-4e4e-a151-aca8a6b2c1af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:22.123524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ccc2a-dcb9-4e4e-a151-aca8a6b2c1af service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:22.129886 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadba427-d488-44a2-9f70-6ca3d27f4fbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:22.130547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadba427-d488-44a2-9f70-6ca3d27f4fbf service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:22.130839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadba427-d488-44a2-9f70-6ca3d27f4fbf service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:22.131119 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadba427-d488-44a2-9f70-6ca3d27f4fbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:22.139512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadba427-d488-44a2-9f70-6ca3d27f4fbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:22.141169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadba427-d488-44a2-9f70-6ca3d27f4fbf service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:22.150004 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1272/2564] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:58:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:58:29.944935 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19eb2a-4599-4f40-b52a-97fb4104f173 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:29.959001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19eb2a-4599-4f40-b52a-97fb4104f173 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:29.966492 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b7281-5365-4b47-8e06-a764f10ebb15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:29.967173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b7281-5365-4b47-8e06-a764f10ebb15 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:29.967457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b7281-5365-4b47-8e06-a764f10ebb15 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:29.967727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b7281-5365-4b47-8e06-a764f10ebb15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:30.031069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b7281-5365-4b47-8e06-a764f10ebb15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:30.032572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b7281-5365-4b47-8e06-a764f10ebb15 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:30.041358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1293/2565] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:32.146308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=54 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:35.171145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97cc0d-cb8e-45b7-9e96-2ca45a24b8ef None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:35.171850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97cc0d-cb8e-45b7-9e96-2ca45a24b8ef None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:35.171850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97cc0d-cb8e-45b7-9e96-2ca45a24b8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:35.189029 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3e97cc0d-cb8e-45b7-9e96-2ca45a24b8ef None None] MFA Rules not processed for user `46d5c3f98d924538a34c4b3a57fcac14`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:35.190263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3e97cc0d-cb8e-45b7-9e96-2ca45a24b8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:35.209459 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1273/2566] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:35.560550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b7281-5365-4b47-8e06-a764f10ebb15 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:35.623450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:35.628750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b7281-5365-4b47-8e06-a764f10ebb15 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:35.633684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:35.634615 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:35.635026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:35.635308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] PATH_INFO: `/v3/users/66f0c40e34fb4e67b24dbf3d1ff5879b` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:35.636731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66f0c40e34fb4e67b24dbf3d1ff5879b)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:35.638804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:35.689484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66f0c40e34fb4e67b24dbf3d1ff5879b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:35.690140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66f0c40e34fb4e67b24dbf3d1ff5879b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:35.705266 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66f0c40e34fb4e67b24dbf3d1ff5879b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:35.723499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66f0c40e34fb4e67b24dbf3d1ff5879b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:35.739598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66f0c40e34fb4e67b24dbf3d1ff5879b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:35.746671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1294/2567] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:58:35 2022] DELETE /identity/v3/users/66f0c40e34fb4e67b24dbf3d1ff5879b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:35.751385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df6ca9-4782-470b-a7f2-f165fa6cd7ea None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:35.751676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df6ca9-4782-470b-a7f2-f165fa6cd7ea None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:35.751949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df6ca9-4782-470b-a7f2-f165fa6cd7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:35.788942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-59df6ca9-4782-470b-a7f2-f165fa6cd7ea None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:35.790658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-59df6ca9-4782-470b-a7f2-f165fa6cd7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:35.867864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1274/2568] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:35.877759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:35.879272 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:35.968137 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925590e4-f3c3-4769-89d9-5e90d8789636 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:35.972664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33745f10-34e1-4390-a6fc-e9466d584bb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:35.973545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33745f10-34e1-4390-a6fc-e9466d584bb9 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:35.974103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33745f10-34e1-4390-a6fc-e9466d584bb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:35.974559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33745f10-34e1-4390-a6fc-e9466d584bb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:35.990795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33745f10-34e1-4390-a6fc-e9466d584bb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:35.996620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33745f10-34e1-4390-a6fc-e9466d584bb9 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:36.021719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1295/2569] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:36.488335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df6ca9-4782-470b-a7f2-f165fa6cd7ea None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:36.490136 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-59df6ca9-4782-470b-a7f2-f165fa6cd7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:36.508313 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df6ca9-4782-470b-a7f2-f165fa6cd7ea None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:36.514606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:36.515504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:36.515792 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:36.516109 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] PATH_INFO: `/v3/projects/534d33178c47478fa57289cdcf2e9448` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:36.517633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=534d33178c47478fa57289cdcf2e9448)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:36.519589 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:36.585621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '534d33178c47478fa57289cdcf2e9448', 'request_id': 'req-cf72066f-17a8-4e0e-93c5-bca70bb1f906'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:36.610653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '534d33178c47478fa57289cdcf2e9448', 'request_id': 'req-cf72066f-17a8-4e0e-93c5-bca70bb1f906'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:36.622034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '534d33178c47478fa57289cdcf2e9448', 'request_id': 'req-cf72066f-17a8-4e0e-93c5-bca70bb1f906'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:36.639753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] The token cache is being invalidate because project 534d33178c47478fa57289cdcf2e9448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:58:36.640339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:36.641075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:36.642909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1275/2570] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:58:36 2022] DELETE /identity/v3/projects/534d33178c47478fa57289cdcf2e9448 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:36.647103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33745f10-34e1-4390-a6fc-e9466d584bb9 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:36.657474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33745f10-34e1-4390-a6fc-e9466d584bb9 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:36.717385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:36.718277 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:36.718556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:36.718841 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] PATH_INFO: `/v3/users/7888af5f06164a5eb1d874ac8ec42d84` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:36.720150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7888af5f06164a5eb1d874ac8ec42d84)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:36.722110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:36.744441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:36.772096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7888af5f06164a5eb1d874ac8ec42d84'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:36.772845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7888af5f06164a5eb1d874ac8ec42d84'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:36.782280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7888af5f06164a5eb1d874ac8ec42d84'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:36.789279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7888af5f06164a5eb1d874ac8ec42d84'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:36.802279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7888af5f06164a5eb1d874ac8ec42d84'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:36.808881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1296/2571] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:58:36 2022] DELETE /identity/v3/users/7888af5f06164a5eb1d874ac8ec42d84 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:37.098552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:37.100523 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:37.116151 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72066f-17a8-4e0e-93c5-bca70bb1f906 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:37.186332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:37.187124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:37.187411 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:37.187690 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] PATH_INFO: `/v3/projects/9e735bc57e7140b3852f6db42c612137` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:37.188902 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e735bc57e7140b3852f6db42c612137)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:37.190966 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:37.212241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:37.243687 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e735bc57e7140b3852f6db42c612137', 'request_id': 'req-5a2c9c1c-9586-484f-8331-0a920d698016'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.267503 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e735bc57e7140b3852f6db42c612137', 'request_id': 'req-5a2c9c1c-9586-484f-8331-0a920d698016'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.292435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e735bc57e7140b3852f6db42c612137', 'request_id': 'req-5a2c9c1c-9586-484f-8331-0a920d698016'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.312708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] The token cache is being invalidate because project 9e735bc57e7140b3852f6db42c612137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:58:37.313162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.313914 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.315954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1276/2572] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:58:37 2022] DELETE /identity/v3/projects/9e735bc57e7140b3852f6db42c612137 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:37.320645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:37.323351 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:37.348576 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7caa0-16c0-4e91-857a-9a5769823d72 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:37.430107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2931b-9493-4314-8482-1cad48309e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:37.430997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2931b-9493-4314-8482-1cad48309e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:37.431284 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2931b-9493-4314-8482-1cad48309e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:37.431563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2931b-9493-4314-8482-1cad48309e38 admin admin] PATH_INFO: `/v3/users/221a3923fda849aeae9af3568793012e` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:37.432781 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2931b-9493-4314-8482-1cad48309e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=221a3923fda849aeae9af3568793012e)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:37.434748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2931b-9493-4314-8482-1cad48309e38 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:37.486423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-22b2931b-9493-4314-8482-1cad48309e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '221a3923fda849aeae9af3568793012e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.487035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-22b2931b-9493-4314-8482-1cad48309e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '221a3923fda849aeae9af3568793012e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.496083 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-22b2931b-9493-4314-8482-1cad48309e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '221a3923fda849aeae9af3568793012e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.503893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-22b2931b-9493-4314-8482-1cad48309e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '221a3923fda849aeae9af3568793012e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.516801 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-22b2931b-9493-4314-8482-1cad48309e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '221a3923fda849aeae9af3568793012e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.523796 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1297/2573] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:58:37 2022] DELETE /identity/v3/users/221a3923fda849aeae9af3568793012e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:37.804976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:37.806507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:37.819476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c9c1c-9586-484f-8331-0a920d698016 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:37.877988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68785f64-e170-4770-9d3f-92b161fc1ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:37.878869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68785f64-e170-4770-9d3f-92b161fc1ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:37.879146 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68785f64-e170-4770-9d3f-92b161fc1ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:37.879418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68785f64-e170-4770-9d3f-92b161fc1ffc admin admin] PATH_INFO: `/v3/projects/44532b16880f4ab4a3ba47587a5204ae` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:37.880671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68785f64-e170-4770-9d3f-92b161fc1ffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=44532b16880f4ab4a3ba47587a5204ae)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:37.882596 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68785f64-e170-4770-9d3f-92b161fc1ffc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:37.930788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-68785f64-e170-4770-9d3f-92b161fc1ffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44532b16880f4ab4a3ba47587a5204ae', 'request_id': 'req-68785f64-e170-4770-9d3f-92b161fc1ffc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.938622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-68785f64-e170-4770-9d3f-92b161fc1ffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44532b16880f4ab4a3ba47587a5204ae', 'request_id': 'req-68785f64-e170-4770-9d3f-92b161fc1ffc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.944258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-68785f64-e170-4770-9d3f-92b161fc1ffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44532b16880f4ab4a3ba47587a5204ae', 'request_id': 'req-68785f64-e170-4770-9d3f-92b161fc1ffc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.955976 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-68785f64-e170-4770-9d3f-92b161fc1ffc admin admin] The token cache is being invalidate because project 44532b16880f4ab4a3ba47587a5204ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:58:37.956296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-68785f64-e170-4770-9d3f-92b161fc1ffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.957106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-68785f64-e170-4770-9d3f-92b161fc1ffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:37.958813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1277/2574] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:58:37 2022] DELETE /identity/v3/projects/44532b16880f4ab4a3ba47587a5204ae => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:37.965031 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbaa76-f27c-49fd-81be-0c9d58ba1386 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:37.965486 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbaa76-f27c-49fd-81be-0c9d58ba1386 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:37.965589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbaa76-f27c-49fd-81be-0c9d58ba1386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:37.985054 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-edcbaa76-f27c-49fd-81be-0c9d58ba1386 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:37.986301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-edcbaa76-f27c-49fd-81be-0c9d58ba1386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:37.999996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:38.054877 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1298/2575] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:38.125452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11a0be-7295-4c12-89d3-78dba0cfa9ff None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.125716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11a0be-7295-4c12-89d3-78dba0cfa9ff None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.126055 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11a0be-7295-4c12-89d3-78dba0cfa9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.150734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-fa11a0be-7295-4c12-89d3-78dba0cfa9ff None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:38.152382 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fa11a0be-7295-4c12-89d3-78dba0cfa9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:38.161526 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1278/2576] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:38.165760 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbaa76-f27c-49fd-81be-0c9d58ba1386 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:38.176072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbaa76-f27c-49fd-81be-0c9d58ba1386 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:38.181279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cd7c0-e9fb-4feb-9a4c-5d9a9b5245a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:38.181939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cd7c0-e9fb-4feb-9a4c-5d9a9b5245a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.182227 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cd7c0-e9fb-4feb-9a4c-5d9a9b5245a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.182514 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cd7c0-e9fb-4feb-9a4c-5d9a9b5245a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.183236 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cd7c0-e9fb-4feb-9a4c-5d9a9b5245a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:58:38.183550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cd7c0-e9fb-4feb-9a4c-5d9a9b5245a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:38.185222 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238cd7c0-e9fb-4feb-9a4c-5d9a9b5245a8 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:38.197606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1299/2577] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:58:38.202151 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11a0be-7295-4c12-89d3-78dba0cfa9ff None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:38.212067 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11a0be-7295-4c12-89d3-78dba0cfa9ff None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:38.217379 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19494e16-df82-4ad8-8454-258cca6c1ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:38.218038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19494e16-df82-4ad8-8454-258cca6c1ffc admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.218322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19494e16-df82-4ad8-8454-258cca6c1ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.218622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19494e16-df82-4ad8-8454-258cca6c1ffc admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.219314 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19494e16-df82-4ad8-8454-258cca6c1ffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:38.221294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19494e16-df82-4ad8-8454-258cca6c1ffc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:38.250075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1279/2578] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:58:38 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:38.254709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cd7c0-e9fb-4feb-9a4c-5d9a9b5245a8 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:38.269010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cd7c0-e9fb-4feb-9a4c-5d9a9b5245a8 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:38.333045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901179fd-7da9-485c-99aa-05760885460f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:38.333740 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901179fd-7da9-485c-99aa-05760885460f admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.334032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901179fd-7da9-485c-99aa-05760885460f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.334288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901179fd-7da9-485c-99aa-05760885460f admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.334973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901179fd-7da9-485c-99aa-05760885460f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:38.339372 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901179fd-7da9-485c-99aa-05760885460f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:38.368267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1300/2579] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:58:38 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 03:58:38.372854 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19494e16-df82-4ad8-8454-258cca6c1ffc admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:38.384256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19494e16-df82-4ad8-8454-258cca6c1ffc admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:38.389796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e5f40-f5c0-499c-a114-33be7f6f9216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:38.390497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e5f40-f5c0-499c-a114-33be7f6f9216 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.390760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e5f40-f5c0-499c-a114-33be7f6f9216 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.391038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e5f40-f5c0-499c-a114-33be7f6f9216 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.391759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e5f40-f5c0-499c-a114-33be7f6f9216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:38.393510 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e5f40-f5c0-499c-a114-33be7f6f9216 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:38.396643 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:38.406931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1280/2580] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:38.410885 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901179fd-7da9-485c-99aa-05760885460f admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:38.426540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901179fd-7da9-485c-99aa-05760885460f admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:38.433724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3b7ee-4152-41da-bf08-a34eb483e933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:38.434538 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3b7ee-4152-41da-bf08-a34eb483e933 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.434664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3b7ee-4152-41da-bf08-a34eb483e933 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.434963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3b7ee-4152-41da-bf08-a34eb483e933 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.435695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3b7ee-4152-41da-bf08-a34eb483e933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:38.437310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3b7ee-4152-41da-bf08-a34eb483e933 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:38.450873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1301/2581] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:38.455051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e5f40-f5c0-499c-a114-33be7f6f9216 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:38.465828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e5f40-f5c0-499c-a114-33be7f6f9216 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:38.475252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e343c-700b-4fbd-84b3-fe395589c9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:38.475252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e343c-700b-4fbd-84b3-fe395589c9be admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.475252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e343c-700b-4fbd-84b3-fe395589c9be admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.475252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e343c-700b-4fbd-84b3-fe395589c9be admin admin] PATH_INFO: `/v3/projects/f4b24d3ae632462da6ba53e147e69e27/users/6e57782bee3a43dda9203c6335ee200e/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.488359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e343c-700b-4fbd-84b3-fe395589c9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b24d3ae632462da6ba53e147e69e27, user_id=6e57782bee3a43dda9203c6335ee200e, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:38.491043 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e343c-700b-4fbd-84b3-fe395589c9be admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:38.504108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1281/2582] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:58:38 2022] PUT /identity/v3/projects/f4b24d3ae632462da6ba53e147e69e27/users/6e57782bee3a43dda9203c6335ee200e/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:38.516190 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66ef1c-5f3a-4eff-a7a0-9d340c5acda0 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.516687 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66ef1c-5f3a-4eff-a7a0-9d340c5acda0 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.516963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66ef1c-5f3a-4eff-a7a0-9d340c5acda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.544500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-1b66ef1c-5f3a-4eff-a7a0-9d340c5acda0 None None] MFA Rules not processed for user `6e57782bee3a43dda9203c6335ee200e`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:38.545888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1b66ef1c-5f3a-4eff-a7a0-9d340c5acda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:38.611303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1302/2583] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:58:38.616201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e343c-700b-4fbd-84b3-fe395589c9be admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:38.682393 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e343c-700b-4fbd-84b3-fe395589c9be admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:38.688297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b41a0-5824-4f35-9ad4-a6cb9a194b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:38.688942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b41a0-5824-4f35-9ad4-a6cb9a194b97 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.689248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b41a0-5824-4f35-9ad4-a6cb9a194b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.689529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b41a0-5824-4f35-9ad4-a6cb9a194b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.690208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b41a0-5824-4f35-9ad4-a6cb9a194b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:38.692209 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b41a0-5824-4f35-9ad4-a6cb9a194b97 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:38.724308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1282/2584] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:58:38 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:38.728822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66ef1c-5f3a-4eff-a7a0-9d340c5acda0 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:38.740445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66ef1c-5f3a-4eff-a7a0-9d340c5acda0 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:38.750009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:38.800414 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a2660-213e-4f39-b2ac-2d4086e317c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:38.801019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a2660-213e-4f39-b2ac-2d4086e317c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.801303 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a2660-213e-4f39-b2ac-2d4086e317c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.801579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a2660-213e-4f39-b2ac-2d4086e317c5 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.802266 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a2660-213e-4f39-b2ac-2d4086e317c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:38.804241 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a2660-213e-4f39-b2ac-2d4086e317c5 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:38.825321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1303/2585] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:58:38 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:38.829398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b41a0-5824-4f35-9ad4-a6cb9a194b97 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:38.840606 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b41a0-5824-4f35-9ad4-a6cb9a194b97 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:38.846437 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6143d-8346-4575-b7f4-725a88f0e0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:38.847108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6143d-8346-4575-b7f4-725a88f0e0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.847401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6143d-8346-4575-b7f4-725a88f0e0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.847672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6143d-8346-4575-b7f4-725a88f0e0b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.848469 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6143d-8346-4575-b7f4-725a88f0e0b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:38.850058 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6143d-8346-4575-b7f4-725a88f0e0b6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:38.861169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1283/2586] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:38.865287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a2660-213e-4f39-b2ac-2d4086e317c5 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:38.875799 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a2660-213e-4f39-b2ac-2d4086e317c5 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:38.881689 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819866db-cd46-4450-bf04-a268c3a4ae5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:38.882575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819866db-cd46-4450-bf04-a268c3a4ae5b admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.882853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819866db-cd46-4450-bf04-a268c3a4ae5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.883149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819866db-cd46-4450-bf04-a268c3a4ae5b admin admin] PATH_INFO: `/v3/projects/3c4269418f014115a75c701061de93e2/users/3154cc34611d4e39b5e8e2cab0a918fd/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.898546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819866db-cd46-4450-bf04-a268c3a4ae5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4269418f014115a75c701061de93e2, user_id=3154cc34611d4e39b5e8e2cab0a918fd, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:38.901274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819866db-cd46-4450-bf04-a268c3a4ae5b admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:38.916654 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1304/2587] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:58:38 2022] PUT /identity/v3/projects/3c4269418f014115a75c701061de93e2/users/3154cc34611d4e39b5e8e2cab0a918fd/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:38.920932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6143d-8346-4575-b7f4-725a88f0e0b6 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:38.931352 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6143d-8346-4575-b7f4-725a88f0e0b6 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:38.991921 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62f105-7f19-49ed-8074-5a34830cb459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:38.992740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62f105-7f19-49ed-8074-5a34830cb459 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:38.993026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62f105-7f19-49ed-8074-5a34830cb459 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:38.993300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62f105-7f19-49ed-8074-5a34830cb459 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:38.994024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62f105-7f19-49ed-8074-5a34830cb459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:38.995655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62f105-7f19-49ed-8074-5a34830cb459 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:39.005625 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1284/2588] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:39.010144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819866db-cd46-4450-bf04-a268c3a4ae5b admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:39.021005 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819866db-cd46-4450-bf04-a268c3a4ae5b admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:39.026361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4e9a9-2bc9-49c6-8023-027105c62920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:39.027660 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4e9a9-2bc9-49c6-8023-027105c62920 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:39.027948 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4e9a9-2bc9-49c6-8023-027105c62920 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:39.028256 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4e9a9-2bc9-49c6-8023-027105c62920 admin admin] PATH_INFO: `/v3/projects/3c4269418f014115a75c701061de93e2/users/3154cc34611d4e39b5e8e2cab0a918fd/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:39.030411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4e9a9-2bc9-49c6-8023-027105c62920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4269418f014115a75c701061de93e2, user_id=3154cc34611d4e39b5e8e2cab0a918fd, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:39.033147 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4e9a9-2bc9-49c6-8023-027105c62920 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:39.043511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1305/2589] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:58:39 2022] PUT /identity/v3/projects/3c4269418f014115a75c701061de93e2/users/3154cc34611d4e39b5e8e2cab0a918fd/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:39.053729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34527e0-998e-4c78-83dc-2f411d93581d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:39.053957 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34527e0-998e-4c78-83dc-2f411d93581d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:39.054227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34527e0-998e-4c78-83dc-2f411d93581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:39.082267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d34527e0-998e-4c78-83dc-2f411d93581d None None] MFA Rules not processed for user `3154cc34611d4e39b5e8e2cab0a918fd`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:39.083462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d34527e0-998e-4c78-83dc-2f411d93581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:39.118287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:39.157400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1285/2590] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:39.169386 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4e9a9-2bc9-49c6-8023-027105c62920 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:39.171130 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c5e4e9a9-2bc9-49c6-8023-027105c62920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:39.230064 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4e9a9-2bc9-49c6-8023-027105c62920 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:39.234113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5fb6a-4474-4a7f-aff5-109f032c9a96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:39.234729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5fb6a-4474-4a7f-aff5-109f032c9a96 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:39.235035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5fb6a-4474-4a7f-aff5-109f032c9a96 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:39.235306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5fb6a-4474-4a7f-aff5-109f032c9a96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:39.245606 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5fb6a-4474-4a7f-aff5-109f032c9a96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:39.247102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5fb6a-4474-4a7f-aff5-109f032c9a96 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:39.256526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1306/2591] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:39.345590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34527e0-998e-4c78-83dc-2f411d93581d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:39.356336 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34527e0-998e-4c78-83dc-2f411d93581d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:39.361220 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67d869-c460-47b3-a609-11b3b32eeb13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:39.361865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67d869-c460-47b3-a609-11b3b32eeb13 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:39.362147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67d869-c460-47b3-a609-11b3b32eeb13 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:39.362414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67d869-c460-47b3-a609-11b3b32eeb13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:39.426738 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67d869-c460-47b3-a609-11b3b32eeb13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:39.428254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67d869-c460-47b3-a609-11b3b32eeb13 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:39.437397 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1286/2592] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:39.727960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5fb6a-4474-4a7f-aff5-109f032c9a96 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:39.793617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5fb6a-4474-4a7f-aff5-109f032c9a96 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:39.798115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:39.799002 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:39.799281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:39.799557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] PATH_INFO: `/v3/users/6e57782bee3a43dda9203c6335ee200e` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:39.800777 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e57782bee3a43dda9203c6335ee200e)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:39.802710 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:39.813106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:39.847977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e57782bee3a43dda9203c6335ee200e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:39.848656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e57782bee3a43dda9203c6335ee200e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:39.856061 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e57782bee3a43dda9203c6335ee200e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:39.863532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e57782bee3a43dda9203c6335ee200e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:39.875678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e57782bee3a43dda9203c6335ee200e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:39.881794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1307/2593] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:58:39 2022] DELETE /identity/v3/users/6e57782bee3a43dda9203c6335ee200e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:39.886627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794197e-626a-4651-b3cf-31913306ec7d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:39.887148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794197e-626a-4651-b3cf-31913306ec7d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:39.887148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794197e-626a-4651-b3cf-31913306ec7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:39.905852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-4794197e-626a-4651-b3cf-31913306ec7d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:39.907044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4794197e-626a-4651-b3cf-31913306ec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:39.968885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1287/2594] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:39.978817 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:39.980380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:40.038966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374a11e-a13b-40f7-940a-a170dabfc1e2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:40.043164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f26fc8-9c25-472b-af97-d30c1603fb2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:40.043808 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f26fc8-9c25-472b-af97-d30c1603fb2b service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:40.044242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f26fc8-9c25-472b-af97-d30c1603fb2b service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:40.044518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f26fc8-9c25-472b-af97-d30c1603fb2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:40.055191 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f26fc8-9c25-472b-af97-d30c1603fb2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:40.056684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f26fc8-9c25-472b-af97-d30c1603fb2b service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:40.067306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1308/2595] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:40.347919 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794197e-626a-4651-b3cf-31913306ec7d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:40.349519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4794197e-626a-4651-b3cf-31913306ec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:40.361892 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794197e-626a-4651-b3cf-31913306ec7d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:40.366795 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:40.367799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:40.368131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:40.368408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] PATH_INFO: `/v3/projects/f4b24d3ae632462da6ba53e147e69e27` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:40.370251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b24d3ae632462da6ba53e147e69e27)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:40.372821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:40.425201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b24d3ae632462da6ba53e147e69e27', 'request_id': 'req-52376735-461e-4c12-8ab4-91c0f3ba47c9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:40.433248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b24d3ae632462da6ba53e147e69e27', 'request_id': 'req-52376735-461e-4c12-8ab4-91c0f3ba47c9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:40.438270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:40.439620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b24d3ae632462da6ba53e147e69e27', 'request_id': 'req-52376735-461e-4c12-8ab4-91c0f3ba47c9'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:40.453878 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] The token cache is being invalidate because project f4b24d3ae632462da6ba53e147e69e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:58:40.454167 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:40.454816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:40.456686 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1288/2596] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:58:40 2022] DELETE /identity/v3/projects/f4b24d3ae632462da6ba53e147e69e27 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:40.460513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f26fc8-9c25-472b-af97-d30c1603fb2b service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:40.470225 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f26fc8-9c25-472b-af97-d30c1603fb2b service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:40.527663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd4c83-a16a-49a0-aaf8-4d46f4c06034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:40.528579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd4c83-a16a-49a0-aaf8-4d46f4c06034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:40.528851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd4c83-a16a-49a0-aaf8-4d46f4c06034 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:40.529126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd4c83-a16a-49a0-aaf8-4d46f4c06034 admin admin] PATH_INFO: `/v3/users/3154cc34611d4e39b5e8e2cab0a918fd` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:40.530355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd4c83-a16a-49a0-aaf8-4d46f4c06034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3154cc34611d4e39b5e8e2cab0a918fd)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:40.532356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd4c83-a16a-49a0-aaf8-4d46f4c06034 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:40.568671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:40.582650 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8bfd4c83-a16a-49a0-aaf8-4d46f4c06034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3154cc34611d4e39b5e8e2cab0a918fd'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:40.583290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8bfd4c83-a16a-49a0-aaf8-4d46f4c06034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3154cc34611d4e39b5e8e2cab0a918fd'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:40.590923 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8bfd4c83-a16a-49a0-aaf8-4d46f4c06034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3154cc34611d4e39b5e8e2cab0a918fd'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:40.598106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8bfd4c83-a16a-49a0-aaf8-4d46f4c06034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3154cc34611d4e39b5e8e2cab0a918fd'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:40.610714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8bfd4c83-a16a-49a0-aaf8-4d46f4c06034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3154cc34611d4e39b5e8e2cab0a918fd'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:40.616860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1309/2597] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:58:40 2022] DELETE /identity/v3/users/3154cc34611d4e39b5e8e2cab0a918fd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:58:40.904709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:40.907040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:40.931012 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52376735-461e-4c12-8ab4-91c0f3ba47c9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:41.002987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb3458-09a5-4dae-a47e-5b0748ab57ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:41.003870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb3458-09a5-4dae-a47e-5b0748ab57ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.004200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb3458-09a5-4dae-a47e-5b0748ab57ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.004482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb3458-09a5-4dae-a47e-5b0748ab57ee admin admin] PATH_INFO: `/v3/projects/3c4269418f014115a75c701061de93e2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.005737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb3458-09a5-4dae-a47e-5b0748ab57ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c4269418f014115a75c701061de93e2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:41.007722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb3458-09a5-4dae-a47e-5b0748ab57ee admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:41.063463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-52eb3458-09a5-4dae-a47e-5b0748ab57ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c4269418f014115a75c701061de93e2', 'request_id': 'req-52eb3458-09a5-4dae-a47e-5b0748ab57ee'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:41.071326 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-52eb3458-09a5-4dae-a47e-5b0748ab57ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c4269418f014115a75c701061de93e2', 'request_id': 'req-52eb3458-09a5-4dae-a47e-5b0748ab57ee'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:41.076896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-52eb3458-09a5-4dae-a47e-5b0748ab57ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c4269418f014115a75c701061de93e2', 'request_id': 'req-52eb3458-09a5-4dae-a47e-5b0748ab57ee'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:41.087823 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-52eb3458-09a5-4dae-a47e-5b0748ab57ee admin admin] The token cache is being invalidate because project 3c4269418f014115a75c701061de93e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:58:41.088139 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-52eb3458-09a5-4dae-a47e-5b0748ab57ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:41.088781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-52eb3458-09a5-4dae-a47e-5b0748ab57ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:41.090264 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1289/2598] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:58:40 2022] DELETE /identity/v3/projects/3c4269418f014115a75c701061de93e2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:41.095466 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72359749-1cbc-4dcc-83cb-b27ccd376e60 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.095973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72359749-1cbc-4dcc-83cb-b27ccd376e60 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.096106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72359749-1cbc-4dcc-83cb-b27ccd376e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.115920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-72359749-1cbc-4dcc-83cb-b27ccd376e60 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:41.117103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-72359749-1cbc-4dcc-83cb-b27ccd376e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:41.178679 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1310/2599] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 03:58:41.187206 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce67b6-067e-403e-9b38-cd9a732284eb None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.187497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce67b6-067e-403e-9b38-cd9a732284eb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.187776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce67b6-067e-403e-9b38-cd9a732284eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.207136 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c7ce67b6-067e-403e-9b38-cd9a732284eb None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:41.208360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c7ce67b6-067e-403e-9b38-cd9a732284eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:41.216966 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1290/2600] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:41.223415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72359749-1cbc-4dcc-83cb-b27ccd376e60 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:41.238501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72359749-1cbc-4dcc-83cb-b27ccd376e60 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:41.245072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ac334-03d1-4264-8b7f-5ef1ad13f493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:41.245726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ac334-03d1-4264-8b7f-5ef1ad13f493 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.246008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ac334-03d1-4264-8b7f-5ef1ad13f493 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.246287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ac334-03d1-4264-8b7f-5ef1ad13f493 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.247012 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ac334-03d1-4264-8b7f-5ef1ad13f493 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:58:41.247310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ac334-03d1-4264-8b7f-5ef1ad13f493 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:41.248977 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ac334-03d1-4264-8b7f-5ef1ad13f493 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:41.260592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1311/2601] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:58:41.264772 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce67b6-067e-403e-9b38-cd9a732284eb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:41.274447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce67b6-067e-403e-9b38-cd9a732284eb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:41.280610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14f48c-2cd8-44c1-8a5a-6287145e57f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:41.281278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14f48c-2cd8-44c1-8a5a-6287145e57f2 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.281543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14f48c-2cd8-44c1-8a5a-6287145e57f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.281815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14f48c-2cd8-44c1-8a5a-6287145e57f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.282495 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14f48c-2cd8-44c1-8a5a-6287145e57f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:41.284455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14f48c-2cd8-44c1-8a5a-6287145e57f2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:41.312166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1291/2602] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:58:41 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:41.319428 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ac334-03d1-4264-8b7f-5ef1ad13f493 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:41.329572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ac334-03d1-4264-8b7f-5ef1ad13f493 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:41.397799 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1fb79b-6ba2-425a-8ef0-cd9d31622211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:41.398382 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1fb79b-6ba2-425a-8ef0-cd9d31622211 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.398668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1fb79b-6ba2-425a-8ef0-cd9d31622211 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.398943 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1fb79b-6ba2-425a-8ef0-cd9d31622211 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.399646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1fb79b-6ba2-425a-8ef0-cd9d31622211 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:41.401605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1fb79b-6ba2-425a-8ef0-cd9d31622211 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:41.423619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1312/2603] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:58:41 2022] POST /identity/v3/users => generated 281 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:41.427887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14f48c-2cd8-44c1-8a5a-6287145e57f2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:41.439055 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14f48c-2cd8-44c1-8a5a-6287145e57f2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:41.443920 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89898794-1274-484e-a8cb-feedf1cfb02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:41.444654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89898794-1274-484e-a8cb-feedf1cfb02d admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.444937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89898794-1274-484e-a8cb-feedf1cfb02d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.445213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89898794-1274-484e-a8cb-feedf1cfb02d admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.446046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89898794-1274-484e-a8cb-feedf1cfb02d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:41.447660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89898794-1274-484e-a8cb-feedf1cfb02d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:41.458126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1292/2604] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:41.462007 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1fb79b-6ba2-425a-8ef0-cd9d31622211 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:41.472404 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1fb79b-6ba2-425a-8ef0-cd9d31622211 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:41.479291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c710005-0642-4996-a678-5014ca60d86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:41.479963 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c710005-0642-4996-a678-5014ca60d86e admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.480363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c710005-0642-4996-a678-5014ca60d86e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.480704 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c710005-0642-4996-a678-5014ca60d86e admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.481760 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c710005-0642-4996-a678-5014ca60d86e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:41.484267 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c710005-0642-4996-a678-5014ca60d86e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:41.495317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1313/2605] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:41.499621 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89898794-1274-484e-a8cb-feedf1cfb02d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:41.510329 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89898794-1274-484e-a8cb-feedf1cfb02d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:41.516277 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f61a5-0785-4449-a053-f175a2d53898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:41.517162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f61a5-0785-4449-a053-f175a2d53898 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.517447 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f61a5-0785-4449-a053-f175a2d53898 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.517715 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f61a5-0785-4449-a053-f175a2d53898 admin admin] PATH_INFO: `/v3/projects/e886799941ea487b83987c783a5257e3/users/c66709fe33c54dc58d200153e109a55f/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.533870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f61a5-0785-4449-a053-f175a2d53898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e886799941ea487b83987c783a5257e3, user_id=c66709fe33c54dc58d200153e109a55f, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:41.536603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f61a5-0785-4449-a053-f175a2d53898 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:41.551741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1293/2606] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:58:41 2022] PUT /identity/v3/projects/e886799941ea487b83987c783a5257e3/users/c66709fe33c54dc58d200153e109a55f/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:41.562191 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497400a3-e478-42ad-a2b7-3ccee8dc4ac4 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.562512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497400a3-e478-42ad-a2b7-3ccee8dc4ac4 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.562797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497400a3-e478-42ad-a2b7-3ccee8dc4ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.589987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-497400a3-e478-42ad-a2b7-3ccee8dc4ac4 None None] MFA Rules not processed for user `c66709fe33c54dc58d200153e109a55f`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:41.591184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-497400a3-e478-42ad-a2b7-3ccee8dc4ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:41.595257 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:41.662612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1314/2607] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:41.694752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f61a5-0785-4449-a053-f175a2d53898 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:41.696991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8b7f61a5-0785-4449-a053-f175a2d53898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:41.718479 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:41.758514 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f61a5-0785-4449-a053-f175a2d53898 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:41.762239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91459cf4-4655-4011-bef5-bb9dfea04bfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:41.762847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91459cf4-4655-4011-bef5-bb9dfea04bfb service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.763128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91459cf4-4655-4011-bef5-bb9dfea04bfb service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.763406 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91459cf4-4655-4011-bef5-bb9dfea04bfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.771925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91459cf4-4655-4011-bef5-bb9dfea04bfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:41.773410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91459cf4-4655-4011-bef5-bb9dfea04bfb service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:41.782954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1294/2608] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:41.797622 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497400a3-e478-42ad-a2b7-3ccee8dc4ac4 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:41.859187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497400a3-e478-42ad-a2b7-3ccee8dc4ac4 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:41.864673 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:41.865559 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.865845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.866121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] PATH_INFO: `/v3/users/c66709fe33c54dc58d200153e109a55f` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.867562 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=c66709fe33c54dc58d200153e109a55f)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:41.869554 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:41.923717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c66709fe33c54dc58d200153e109a55f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:41.924327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c66709fe33c54dc58d200153e109a55f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:41.933355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c66709fe33c54dc58d200153e109a55f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:41.941172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c66709fe33c54dc58d200153e109a55f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:41.956522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c66709fe33c54dc58d200153e109a55f'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:41.963038 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1315/2609] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:58:41 2022] DELETE /identity/v3/users/c66709fe33c54dc58d200153e109a55f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:41.967645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae079f-5441-4cd2-a52e-b8dfe3896ad0 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:41.968155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae079f-5441-4cd2-a52e-b8dfe3896ad0 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:41.968337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae079f-5441-4cd2-a52e-b8dfe3896ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:41.986400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-92ae079f-5441-4cd2-a52e-b8dfe3896ad0 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:41.987597 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-92ae079f-5441-4cd2-a52e-b8dfe3896ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:42.050847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1295/2610] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:42.060611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:42.062100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:42.090358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:42.120937 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250cd24-695e-4e12-ab1f-931a8bb2e7ab admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:42.124729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443751c-10d9-4438-886c-b6b76c53f1e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:42.125380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443751c-10d9-4438-886c-b6b76c53f1e0 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:42.125760 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443751c-10d9-4438-886c-b6b76c53f1e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:42.126034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443751c-10d9-4438-886c-b6b76c53f1e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:42.135411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443751c-10d9-4438-886c-b6b76c53f1e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:42.136893 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443751c-10d9-4438-886c-b6b76c53f1e0 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:42.146278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1316/2611] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:42.422507 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae079f-5441-4cd2-a52e-b8dfe3896ad0 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:42.424131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-92ae079f-5441-4cd2-a52e-b8dfe3896ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:42.437850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ae079f-5441-4cd2-a52e-b8dfe3896ad0 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:42.442879 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa177f-8043-4e9a-9a16-8f606d3add73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:42.443795 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa177f-8043-4e9a-9a16-8f606d3add73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:42.444105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa177f-8043-4e9a-9a16-8f606d3add73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:42.444387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa177f-8043-4e9a-9a16-8f606d3add73 admin admin] PATH_INFO: `/v3/projects/e886799941ea487b83987c783a5257e3` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:42.445778 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa177f-8043-4e9a-9a16-8f606d3add73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e886799941ea487b83987c783a5257e3)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:42.447735 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa177f-8043-4e9a-9a16-8f606d3add73 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:42.494734 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f1fa177f-8043-4e9a-9a16-8f606d3add73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e886799941ea487b83987c783a5257e3', 'request_id': 'req-f1fa177f-8043-4e9a-9a16-8f606d3add73'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:42.502307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f1fa177f-8043-4e9a-9a16-8f606d3add73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e886799941ea487b83987c783a5257e3', 'request_id': 'req-f1fa177f-8043-4e9a-9a16-8f606d3add73'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:42.508469 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f1fa177f-8043-4e9a-9a16-8f606d3add73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e886799941ea487b83987c783a5257e3', 'request_id': 'req-f1fa177f-8043-4e9a-9a16-8f606d3add73'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:42.520086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f1fa177f-8043-4e9a-9a16-8f606d3add73 admin admin] The token cache is being invalidate because project e886799941ea487b83987c783a5257e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:58:42.520337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f1fa177f-8043-4e9a-9a16-8f606d3add73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:42.521041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-f1fa177f-8043-4e9a-9a16-8f606d3add73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:42.522680 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1296/2612] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:58:42 2022] DELETE /identity/v3/projects/e886799941ea487b83987c783a5257e3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:42.529324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe0a39-2a71-4dab-b7fb-55cd56e90d5f None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:42.529610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe0a39-2a71-4dab-b7fb-55cd56e90d5f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:42.529882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe0a39-2a71-4dab-b7fb-55cd56e90d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:42.548598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-babe0a39-2a71-4dab-b7fb-55cd56e90d5f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:42.549789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-babe0a39-2a71-4dab-b7fb-55cd56e90d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:42.612156 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1317/2613] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:42.621102 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336a5e4-3d8f-4daf-a538-c181f4ddaade None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:42.621396 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336a5e4-3d8f-4daf-a538-c181f4ddaade None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:42.621676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336a5e4-3d8f-4daf-a538-c181f4ddaade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:42.644460 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1336a5e4-3d8f-4daf-a538-c181f4ddaade None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:42.645876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1336a5e4-3d8f-4daf-a538-c181f4ddaade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:42.656905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1297/2614] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:42.661518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe0a39-2a71-4dab-b7fb-55cd56e90d5f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:42.672113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe0a39-2a71-4dab-b7fb-55cd56e90d5f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:42.677877 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab0b40-1d9a-409c-ac5c-612316a27e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:42.678551 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab0b40-1d9a-409c-ac5c-612316a27e3c admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:42.678835 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab0b40-1d9a-409c-ac5c-612316a27e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:42.679110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab0b40-1d9a-409c-ac5c-612316a27e3c admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:42.679933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab0b40-1d9a-409c-ac5c-612316a27e3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:58:42.680277 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab0b40-1d9a-409c-ac5c-612316a27e3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:42.681920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab0b40-1d9a-409c-ac5c-612316a27e3c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:42.700953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1318/2615] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:58:42.706027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336a5e4-3d8f-4daf-a538-c181f4ddaade None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:42.717732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336a5e4-3d8f-4daf-a538-c181f4ddaade None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:42.723705 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dfb29c-b04b-4844-b25b-5a82cabdcd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:42.724399 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfb29c-b04b-4844-b25b-5a82cabdcd65 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:42.724682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfb29c-b04b-4844-b25b-5a82cabdcd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:42.724954 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfb29c-b04b-4844-b25b-5a82cabdcd65 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:42.725638 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfb29c-b04b-4844-b25b-5a82cabdcd65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:42.727592 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfb29c-b04b-4844-b25b-5a82cabdcd65 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:42.756229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1298/2616] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:58:42 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:42.760333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab0b40-1d9a-409c-ac5c-612316a27e3c admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:42.771649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab0b40-1d9a-409c-ac5c-612316a27e3c admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:42.833317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb585177-25e8-4ea9-a519-ba6ec13f694e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:42.833995 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb585177-25e8-4ea9-a519-ba6ec13f694e admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:42.834268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb585177-25e8-4ea9-a519-ba6ec13f694e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:42.834552 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb585177-25e8-4ea9-a519-ba6ec13f694e admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:42.835238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb585177-25e8-4ea9-a519-ba6ec13f694e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:42.837247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb585177-25e8-4ea9-a519-ba6ec13f694e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:42.860457 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1319/2617] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:58:42 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:58:42.865022 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dfb29c-b04b-4844-b25b-5a82cabdcd65 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:42.875811 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dfb29c-b04b-4844-b25b-5a82cabdcd65 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:42.880696 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadd1d5-8227-4270-8efb-d4b3f5334e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:42.881371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd1d5-8227-4270-8efb-d4b3f5334e8a admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:42.881657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd1d5-8227-4270-8efb-d4b3f5334e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:42.881937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd1d5-8227-4270-8efb-d4b3f5334e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:42.882698 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadd1d5-8227-4270-8efb-d4b3f5334e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:42.884438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadd1d5-8227-4270-8efb-d4b3f5334e8a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:42.894714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1299/2618] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:42.898789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb585177-25e8-4ea9-a519-ba6ec13f694e admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:42.909144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb585177-25e8-4ea9-a519-ba6ec13f694e admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:42.914172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c013bf-bdc2-47f1-95e7-fea79c50acfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:42.914863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c013bf-bdc2-47f1-95e7-fea79c50acfb admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:42.915151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c013bf-bdc2-47f1-95e7-fea79c50acfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:42.915425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c013bf-bdc2-47f1-95e7-fea79c50acfb admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:42.916186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c013bf-bdc2-47f1-95e7-fea79c50acfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:42.917788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c013bf-bdc2-47f1-95e7-fea79c50acfb admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:42.927750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1320/2619] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:58:42.931801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadd1d5-8227-4270-8efb-d4b3f5334e8a admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:42.939874 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:42.942379 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadd1d5-8227-4270-8efb-d4b3f5334e8a admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:42.948486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f912c-fb61-4db8-a818-0c13c4a70085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:42.950007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f912c-fb61-4db8-a818-0c13c4a70085 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:42.950635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f912c-fb61-4db8-a818-0c13c4a70085 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:42.951089 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f912c-fb61-4db8-a818-0c13c4a70085 admin admin] PATH_INFO: `/v3/projects/ad8c8a8ff8f545ca8ff374e182e8e968/users/f205273cb211487c9155335a19fd1b83/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:42.968170 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f912c-fb61-4db8-a818-0c13c4a70085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8c8a8ff8f545ca8ff374e182e8e968, user_id=f205273cb211487c9155335a19fd1b83, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:42.970900 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f912c-fb61-4db8-a818-0c13c4a70085 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:42.987299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1300/2620] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:58:42 2022] PUT /identity/v3/projects/ad8c8a8ff8f545ca8ff374e182e8e968/users/f205273cb211487c9155335a19fd1b83/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 03:58:42.998027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5c4fd-3af3-45e6-b9da-8f34cdf379f5 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:42.998324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5c4fd-3af3-45e6-b9da-8f34cdf379f5 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:42.998612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5c4fd-3af3-45e6-b9da-8f34cdf379f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:43.034004 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-08c5c4fd-3af3-45e6-b9da-8f34cdf379f5 None None] MFA Rules not processed for user `f205273cb211487c9155335a19fd1b83`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:43.035161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-08c5c4fd-3af3-45e6-b9da-8f34cdf379f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:43.060182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:43.110790 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1321/2621] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:43.125615 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f912c-fb61-4db8-a818-0c13c4a70085 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:43.127733 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6c3f912c-fb61-4db8-a818-0c13c4a70085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:43.189184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f912c-fb61-4db8-a818-0c13c4a70085 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:43.193073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55d001-5638-4c5f-bac3-c820f31b8a5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:43.193727 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d001-5638-4c5f-bac3-c820f31b8a5a service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:43.194012 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d001-5638-4c5f-bac3-c820f31b8a5a service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:43.194296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55d001-5638-4c5f-bac3-c820f31b8a5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:43.205922 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55d001-5638-4c5f-bac3-c820f31b8a5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:43.207390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55d001-5638-4c5f-bac3-c820f31b8a5a service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:43.217797 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1301/2622] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:45.834485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5c4fd-3af3-45e6-b9da-8f34cdf379f5 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:45.898213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5c4fd-3af3-45e6-b9da-8f34cdf379f5 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:45.903238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:45.904154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:45.904437 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:45.904688 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] PATH_INFO: `/v3/users/f205273cb211487c9155335a19fd1b83` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:45.905892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f205273cb211487c9155335a19fd1b83)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:45.907827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:45.966277 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f205273cb211487c9155335a19fd1b83'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:45.966962 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f205273cb211487c9155335a19fd1b83'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:45.975272 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f205273cb211487c9155335a19fd1b83'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:45.983565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f205273cb211487c9155335a19fd1b83'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:45.996760 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f205273cb211487c9155335a19fd1b83'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:46.003489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1322/2623] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:58:45 2022] DELETE /identity/v3/users/f205273cb211487c9155335a19fd1b83 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:46.008269 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b042b-0f66-4d3d-ba86-8f187f9a7117 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:46.008832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b042b-0f66-4d3d-ba86-8f187f9a7117 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:46.008948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b042b-0f66-4d3d-ba86-8f187f9a7117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:46.027782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-701b042b-0f66-4d3d-ba86-8f187f9a7117 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:46.028949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-701b042b-0f66-4d3d-ba86-8f187f9a7117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:46.095595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1302/2624] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:46.105676 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:46.107184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:46.154117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:46.169298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131176d-9e5c-40d6-8f48-54740da0daef admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:46.173307 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece978c-4641-45d2-a5e5-a6063211791d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:46.173974 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece978c-4641-45d2-a5e5-a6063211791d service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:46.174397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece978c-4641-45d2-a5e5-a6063211791d service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:46.174692 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece978c-4641-45d2-a5e5-a6063211791d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:46.185178 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece978c-4641-45d2-a5e5-a6063211791d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:46.186694 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece978c-4641-45d2-a5e5-a6063211791d service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:46.198152 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1323/2625] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:58:46.473561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b042b-0f66-4d3d-ba86-8f187f9a7117 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:46.475006 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-701b042b-0f66-4d3d-ba86-8f187f9a7117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:46.488046 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b042b-0f66-4d3d-ba86-8f187f9a7117 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:46.494557 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:46.496255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:46.496722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:46.497227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] PATH_INFO: `/v3/projects/ad8c8a8ff8f545ca8ff374e182e8e968` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:46.499337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8c8a8ff8f545ca8ff374e182e8e968)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:46.501818 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:46.559024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8c8a8ff8f545ca8ff374e182e8e968', 'request_id': 'req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:46.566533 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8c8a8ff8f545ca8ff374e182e8e968', 'request_id': 'req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:46.573058 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8c8a8ff8f545ca8ff374e182e8e968', 'request_id': 'req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:46.584356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] The token cache is being invalidate because project ad8c8a8ff8f545ca8ff374e182e8e968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:58:46.584644 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:46.585288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:58:46.587226 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1303/2626] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:58:46 2022] DELETE /identity/v3/projects/ad8c8a8ff8f545ca8ff374e182e8e968 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:58:50.096297 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c337a-6cb0-40a5-aa53-5fcd9bbea489 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:50.096788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c337a-6cb0-40a5-aa53-5fcd9bbea489 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:50.096910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c337a-6cb0-40a5-aa53-5fcd9bbea489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:50.115030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-531c337a-6cb0-40a5-aa53-5fcd9bbea489 None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:58:50.116261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-531c337a-6cb0-40a5-aa53-5fcd9bbea489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:50.175498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1324/2627] 10.222.0.20 () {58 vars in 967 bytes} [Tue Jul 12 03:58:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:58:50.188303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:50.189777 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:50.200280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:58:50.268269 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bc0e1-1e0c-4747-92c2-1d88a11d1b2d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:50.273574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac98e9c-ed2b-41ce-a4a3-3879ea3ac4f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:50.274321 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac98e9c-ed2b-41ce-a4a3-3879ea3ac4f5 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:50.274608 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac98e9c-ed2b-41ce-a4a3-3879ea3ac4f5 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:50.274882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac98e9c-ed2b-41ce-a4a3-3879ea3ac4f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:50.283935 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac98e9c-ed2b-41ce-a4a3-3879ea3ac4f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:50.285582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac98e9c-ed2b-41ce-a4a3-3879ea3ac4f5 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:50.293812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1304/2628] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:58:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 03:58:59.809113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c337a-6cb0-40a5-aa53-5fcd9bbea489 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:58:59.810702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-531c337a-6cb0-40a5-aa53-5fcd9bbea489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:59.874564 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c337a-6cb0-40a5-aa53-5fcd9bbea489 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:58:59.878575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951fb64-e64c-473a-afc5-dadb885f6f6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:58:59.879220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951fb64-e64c-473a-afc5-dadb885f6f6f service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:58:59.879505 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951fb64-e64c-473a-afc5-dadb885f6f6f service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:58:59.879803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951fb64-e64c-473a-afc5-dadb885f6f6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:58:59.881692 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0951fb64-e64c-473a-afc5-dadb885f6f6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:58:59.956504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951fb64-e64c-473a-afc5-dadb885f6f6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:58:59.957965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951fb64-e64c-473a-afc5-dadb885f6f6f service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:58:59.968472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1325/2629] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:58:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:59:00.290634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=29 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:59:05.911255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac98e9c-ed2b-41ce-a4a3-3879ea3ac4f5 service swift] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:05.912846 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9ac98e9c-ed2b-41ce-a4a3-3879ea3ac4f5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:05.976389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac98e9c-ed2b-41ce-a4a3-3879ea3ac4f5 service swift] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:05.980133 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56643801-5c0c-4b8b-acfd-a3df67dee756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:05.980810 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56643801-5c0c-4b8b-acfd-a3df67dee756 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:05.981075 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56643801-5c0c-4b8b-acfd-a3df67dee756 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:05.981350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56643801-5c0c-4b8b-acfd-a3df67dee756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:05.983114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-56643801-5c0c-4b8b-acfd-a3df67dee756 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:06.057007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56643801-5c0c-4b8b-acfd-a3df67dee756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:06.058462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56643801-5c0c-4b8b-acfd-a3df67dee756 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:06.069573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1305/2630] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:59:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:59:09.963725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=82 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:59:11.030083 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed434a8-f023-4e4c-a16c-ebf2c30ea20b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:11.030645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed434a8-f023-4e4c-a16c-ebf2c30ea20b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:11.030645 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed434a8-f023-4e4c-a16c-ebf2c30ea20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:11.049208 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8ed434a8-f023-4e4c-a16c-ebf2c30ea20b None None] MFA Rules not processed for user `2b01c3b5c3384d4ca0999e9156c3a9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:59:11.050422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8ed434a8-f023-4e4c-a16c-ebf2c30ea20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:11.058921 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1326/2631] 10.222.0.20 () {58 vars in 967 bytes} [Tue Jul 12 03:59:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:59:11.081256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56643801-5c0c-4b8b-acfd-a3df67dee756 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:11.093321 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56643801-5c0c-4b8b-acfd-a3df67dee756 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:11.098765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bed2c-7de2-4ff8-932d-141ae0ae5db5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0026b8f11158492b90da3395cc2c43c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:11.099430 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bed2c-7de2-4ff8-932d-141ae0ae5db5 service swift] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:11.099710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bed2c-7de2-4ff8-932d-141ae0ae5db5 service swift] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:11.099978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bed2c-7de2-4ff8-932d-141ae0ae5db5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:11.108712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bed2c-7de2-4ff8-932d-141ae0ae5db5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:11.110315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bed2c-7de2-4ff8-932d-141ae0ae5db5 service swift] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:11.118701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1306/2632] 10.222.0.20 () {60 vars in 1384 bytes} [Tue Jul 12 03:59:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 03:59:18.089608 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed434a8-f023-4e4c-a16c-ebf2c30ea20b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:18.091121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8ed434a8-f023-4e4c-a16c-ebf2c30ea20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:18.161035 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed434a8-f023-4e4c-a16c-ebf2c30ea20b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:18.165714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:18.166604 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:18.166950 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:18.167370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] PATH_INFO: `/v3/users/8701284390d345eabc027e05f9ac262b` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:18.168532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8701284390d345eabc027e05f9ac262b)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:18.170473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:18.220698 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8701284390d345eabc027e05f9ac262b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.221339 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8701284390d345eabc027e05f9ac262b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.230691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8701284390d345eabc027e05f9ac262b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.237942 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8701284390d345eabc027e05f9ac262b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.249701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8701284390d345eabc027e05f9ac262b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.255755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1327/2633] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:59:18 2022] DELETE /identity/v3/users/8701284390d345eabc027e05f9ac262b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:59:18.433002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bed2c-7de2-4ff8-932d-141ae0ae5db5 service swift] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:18.445297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bed2c-7de2-4ff8-932d-141ae0ae5db5 service swift] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:18.505405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:18.506281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:18.506559 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:18.506833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] PATH_INFO: `/v3/projects/b38d912121c64d5e887dc21e354b3cfd` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:18.508032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38d912121c64d5e887dc21e354b3cfd)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:18.510056 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:18.546506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:59:18.559943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38d912121c64d5e887dc21e354b3cfd', 'request_id': 'req-9481b078-dec9-4a2a-9671-594d59959450'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.568304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38d912121c64d5e887dc21e354b3cfd', 'request_id': 'req-9481b078-dec9-4a2a-9671-594d59959450'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.574178 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38d912121c64d5e887dc21e354b3cfd', 'request_id': 'req-9481b078-dec9-4a2a-9671-594d59959450'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.585160 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] The token cache is being invalidate because project b38d912121c64d5e887dc21e354b3cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:59:18.585445 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.586050 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.587611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1307/2634] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:59:18 2022] DELETE /identity/v3/projects/b38d912121c64d5e887dc21e354b3cfd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:59:18.592093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:18.593607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:18.607753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecc11b-b0cd-4cca-bf69-7c285c223ea2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:18.671521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039510f-c13a-471c-93c7-e6884925d126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:18.672427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039510f-c13a-471c-93c7-e6884925d126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:18.672744 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039510f-c13a-471c-93c7-e6884925d126 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:18.673017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039510f-c13a-471c-93c7-e6884925d126 admin admin] PATH_INFO: `/v3/users/f9eefffeaa3640219185515c6b3efbaf` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:18.674325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039510f-c13a-471c-93c7-e6884925d126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9eefffeaa3640219185515c6b3efbaf)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:18.676383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039510f-c13a-471c-93c7-e6884925d126 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:18.692861 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:59:18.728579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4039510f-c13a-471c-93c7-e6884925d126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9eefffeaa3640219185515c6b3efbaf'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.729231 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4039510f-c13a-471c-93c7-e6884925d126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9eefffeaa3640219185515c6b3efbaf'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.737588 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4039510f-c13a-471c-93c7-e6884925d126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9eefffeaa3640219185515c6b3efbaf'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.745404 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4039510f-c13a-471c-93c7-e6884925d126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9eefffeaa3640219185515c6b3efbaf'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.757413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-4039510f-c13a-471c-93c7-e6884925d126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9eefffeaa3640219185515c6b3efbaf'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:18.764063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1328/2635] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 03:59:18 2022] DELETE /identity/v3/users/f9eefffeaa3640219185515c6b3efbaf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:59:18.953434 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:18.955030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:18.968670 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481b078-dec9-4a2a-9671-594d59959450 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:19.029271 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:19.030070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:19.030351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:19.030630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2 admin admin] PATH_INFO: `/v3/projects/e8a04014e6344f8fa9145b52d03679fa` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:19.031838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8a04014e6344f8fa9145b52d03679fa)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:19.033942 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:19.085187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8a04014e6344f8fa9145b52d03679fa', 'request_id': 'req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:19.092895 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8a04014e6344f8fa9145b52d03679fa', 'request_id': 'req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:19.098660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8a04014e6344f8fa9145b52d03679fa', 'request_id': 'req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:19.110420 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2 admin admin] The token cache is being invalidate because project e8a04014e6344f8fa9145b52d03679fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 03:59:19.110707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:19.111420 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-ce959d1a-ac47-44d7-b5d8-76203fdc19f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 03:59:19.113106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1308/2636] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 03:59:18 2022] DELETE /identity/v3/projects/e8a04014e6344f8fa9145b52d03679fa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:59:19.118553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb38de-f671-492f-8f25-76138adafc5f None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:19.119044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb38de-f671-492f-8f25-76138adafc5f None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:19.119153 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb38de-f671-492f-8f25-76138adafc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:19.139086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-88eb38de-f671-492f-8f25-76138adafc5f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:59:19.140552 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-88eb38de-f671-492f-8f25-76138adafc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:19.207219 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1329/2637] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:59:19.215875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47710e-98c0-4412-afce-55b37c1551f3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:19.216200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47710e-98c0-4412-afce-55b37c1551f3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:19.216479 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47710e-98c0-4412-afce-55b37c1551f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:19.238293 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3c47710e-98c0-4412-afce-55b37c1551f3 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:59:19.239917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3c47710e-98c0-4412-afce-55b37c1551f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:19.250124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1309/2638] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:59:19.254459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb38de-f671-492f-8f25-76138adafc5f None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:19.264828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb38de-f671-492f-8f25-76138adafc5f None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:19.269665 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093372a-61ec-4b8c-800c-dcbf410f508a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:19.270319 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093372a-61ec-4b8c-800c-dcbf410f508a admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:19.270599 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093372a-61ec-4b8c-800c-dcbf410f508a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:19.270869 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093372a-61ec-4b8c-800c-dcbf410f508a admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:19.271597 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093372a-61ec-4b8c-800c-dcbf410f508a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 03:59:19.271898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093372a-61ec-4b8c-800c-dcbf410f508a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:19.273553 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093372a-61ec-4b8c-800c-dcbf410f508a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:19.286142 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1330/2639] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 03:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 03:59:19.290372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47710e-98c0-4412-afce-55b37c1551f3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:19.299737 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47710e-98c0-4412-afce-55b37c1551f3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:19.304916 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4c11a-2324-4ccb-a944-6c0076ef743b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:19.305564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4c11a-2324-4ccb-a944-6c0076ef743b admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:19.305841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4c11a-2324-4ccb-a944-6c0076ef743b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:19.306110 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4c11a-2324-4ccb-a944-6c0076ef743b admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:19.306801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4c11a-2324-4ccb-a944-6c0076ef743b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:19.308870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4c11a-2324-4ccb-a944-6c0076ef743b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:19.347287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1310/2640] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:59:19 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:59:19.351393 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093372a-61ec-4b8c-800c-dcbf410f508a admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:19.361750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093372a-61ec-4b8c-800c-dcbf410f508a admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:19.418798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd764020-7684-4eb7-a8ef-04708205b47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:19.419459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd764020-7684-4eb7-a8ef-04708205b47e admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:19.419742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd764020-7684-4eb7-a8ef-04708205b47e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:19.420045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd764020-7684-4eb7-a8ef-04708205b47e admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:19.420737 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd764020-7684-4eb7-a8ef-04708205b47e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:19.422677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd764020-7684-4eb7-a8ef-04708205b47e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:19.444077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1331/2641] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:59:19 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:59:19.448296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4c11a-2324-4ccb-a944-6c0076ef743b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:19.459056 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4c11a-2324-4ccb-a944-6c0076ef743b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:19.464830 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924187f-850c-477d-9897-247cc931cda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:19.465497 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924187f-850c-477d-9897-247cc931cda5 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:19.465782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924187f-850c-477d-9897-247cc931cda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:19.466057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924187f-850c-477d-9897-247cc931cda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:19.466788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924187f-850c-477d-9897-247cc931cda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:19.468464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924187f-850c-477d-9897-247cc931cda5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:19.473874 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:59:19.479131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1311/2642] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:59:19.483276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd764020-7684-4eb7-a8ef-04708205b47e admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:19.496146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd764020-7684-4eb7-a8ef-04708205b47e admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:19.501622 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70620e9-fe2e-4cd0-8aa3-79e35f9d635a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:19.502293 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70620e9-fe2e-4cd0-8aa3-79e35f9d635a admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:19.502581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70620e9-fe2e-4cd0-8aa3-79e35f9d635a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:19.502856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70620e9-fe2e-4cd0-8aa3-79e35f9d635a admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:19.503620 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70620e9-fe2e-4cd0-8aa3-79e35f9d635a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:19.505252 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70620e9-fe2e-4cd0-8aa3-79e35f9d635a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:19.507608 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:59:19.517711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1332/2643] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:59:19.522328 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924187f-850c-477d-9897-247cc931cda5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:19.532925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924187f-850c-477d-9897-247cc931cda5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:19.537881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c90d4-4832-4822-93d7-4b4aebd81b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:19.538716 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c90d4-4832-4822-93d7-4b4aebd81b5e admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:19.539005 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c90d4-4832-4822-93d7-4b4aebd81b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:19.539281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c90d4-4832-4822-93d7-4b4aebd81b5e admin admin] PATH_INFO: `/v3/projects/aac1a71ff09f4ae38a5c3704fc43a950/users/5991e17e8dfb4a59a4dd1a4da24ff90c/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:19.554894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c90d4-4832-4822-93d7-4b4aebd81b5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac1a71ff09f4ae38a5c3704fc43a950, user_id=5991e17e8dfb4a59a4dd1a4da24ff90c, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:19.557664 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c90d4-4832-4822-93d7-4b4aebd81b5e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:19.572890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1312/2644] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:59:19 2022] PUT /identity/v3/projects/aac1a71ff09f4ae38a5c3704fc43a950/users/5991e17e8dfb4a59a4dd1a4da24ff90c/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:59:19.578163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682c683-9c5b-40ff-9e4f-a035f54108ab None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:19.578454 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682c683-9c5b-40ff-9e4f-a035f54108ab None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:19.578753 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682c683-9c5b-40ff-9e4f-a035f54108ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:19.596824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-6682c683-9c5b-40ff-9e4f-a035f54108ab None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:59:19.598010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-6682c683-9c5b-40ff-9e4f-a035f54108ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:19.659719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1333/2645] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:59:19.670972 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c90d4-4832-4822-93d7-4b4aebd81b5e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:19.673083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ec4c90d4-4832-4822-93d7-4b4aebd81b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:19.732626 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c90d4-4832-4822-93d7-4b4aebd81b5e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:19.736692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae818acf-0343-48b9-a17d-6c4d3aadb0b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:19.737343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae818acf-0343-48b9-a17d-6c4d3aadb0b7 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:19.737631 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae818acf-0343-48b9-a17d-6c4d3aadb0b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:19.737908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae818acf-0343-48b9-a17d-6c4d3aadb0b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:19.747431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae818acf-0343-48b9-a17d-6c4d3aadb0b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:19.748990 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae818acf-0343-48b9-a17d-6c4d3aadb0b7 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:19.758941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1313/2646] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:59:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:59:24.107828 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf3170-d6fd-4117-8448-b709e33c8c28 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:24.108455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf3170-d6fd-4117-8448-b709e33c8c28 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:24.108455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf3170-d6fd-4117-8448-b709e33c8c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:24.137470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-5dbf3170-d6fd-4117-8448-b709e33c8c28 None None] MFA Rules not processed for user `5991e17e8dfb4a59a4dd1a4da24ff90c`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:59:24.138767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5dbf3170-d6fd-4117-8448-b709e33c8c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:24.203649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1334/2647] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:59:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:59:24.208432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae818acf-0343-48b9-a17d-6c4d3aadb0b7 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:24.218885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae818acf-0343-48b9-a17d-6c4d3aadb0b7 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:24.224034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f246f-7343-47a4-8f4a-fae1d4b4de0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:24.224750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f246f-7343-47a4-8f4a-fae1d4b4de0b admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:24.225035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f246f-7343-47a4-8f4a-fae1d4b4de0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:24.225303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f246f-7343-47a4-8f4a-fae1d4b4de0b admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:24.226015 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f246f-7343-47a4-8f4a-fae1d4b4de0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:24.227992 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f246f-7343-47a4-8f4a-fae1d4b4de0b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:24.256532 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1314/2648] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 03:59:24 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:59:24.260887 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf3170-d6fd-4117-8448-b709e33c8c28 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:24.284902 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf3170-d6fd-4117-8448-b709e33c8c28 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:24.363843 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888b193-b165-4c60-89d5-2a1e4b6ddfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:24.364579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888b193-b165-4c60-89d5-2a1e4b6ddfdb admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:24.364860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888b193-b165-4c60-89d5-2a1e4b6ddfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:24.365171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888b193-b165-4c60-89d5-2a1e4b6ddfdb admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:24.365868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888b193-b165-4c60-89d5-2a1e4b6ddfdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:24.367818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888b193-b165-4c60-89d5-2a1e4b6ddfdb admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:24.392203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1335/2649] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 03:59:24 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 03:59:24.395779 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f246f-7343-47a4-8f4a-fae1d4b4de0b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:24.407714 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f246f-7343-47a4-8f4a-fae1d4b4de0b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:24.413374 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01188da7-50fd-4300-b681-12aea83c0d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:24.414695 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01188da7-50fd-4300-b681-12aea83c0d37 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:24.414991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01188da7-50fd-4300-b681-12aea83c0d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:24.415299 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01188da7-50fd-4300-b681-12aea83c0d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:24.416081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01188da7-50fd-4300-b681-12aea83c0d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:24.417841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01188da7-50fd-4300-b681-12aea83c0d37 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:24.430631 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1315/2650] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:59:24.434833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888b193-b165-4c60-89d5-2a1e4b6ddfdb admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:24.445442 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888b193-b165-4c60-89d5-2a1e4b6ddfdb admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:24.450481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e176b26-e005-4e8b-ab8e-2605b5b68836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:24.451397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e176b26-e005-4e8b-ab8e-2605b5b68836 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:24.451677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e176b26-e005-4e8b-ab8e-2605b5b68836 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:24.451951 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e176b26-e005-4e8b-ab8e-2605b5b68836 admin admin] PATH_INFO: `/v3/projects/9340450b91db44459cc62a2276957230/users/525f7f33ec554dffbdad5d372805d0b2/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:24.468094 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e176b26-e005-4e8b-ab8e-2605b5b68836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9340450b91db44459cc62a2276957230, user_id=525f7f33ec554dffbdad5d372805d0b2, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:24.470810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e176b26-e005-4e8b-ab8e-2605b5b68836 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:24.473242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:59:24.485613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1336/2651] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:59:24 2022] PUT /identity/v3/projects/9340450b91db44459cc62a2276957230/users/525f7f33ec554dffbdad5d372805d0b2/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:59:24.489619 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01188da7-50fd-4300-b681-12aea83c0d37 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:24.500846 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01188da7-50fd-4300-b681-12aea83c0d37 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:24.559951 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e3772-0ffc-4778-a61e-ae7974726ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:24.560773 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e3772-0ffc-4778-a61e-ae7974726ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:24.561070 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e3772-0ffc-4778-a61e-ae7974726ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:24.561327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e3772-0ffc-4778-a61e-ae7974726ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:24.562066 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e3772-0ffc-4778-a61e-ae7974726ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:24.563671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e3772-0ffc-4778-a61e-ae7974726ef4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:24.574453 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1316/2652] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 03:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 03:59:24.578701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e176b26-e005-4e8b-ab8e-2605b5b68836 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:24.589485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e176b26-e005-4e8b-ab8e-2605b5b68836 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:24.595056 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2fd1c6-cddc-494d-b84b-5dc9c361a34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:24.595965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fd1c6-cddc-494d-b84b-5dc9c361a34f admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:24.596296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fd1c6-cddc-494d-b84b-5dc9c361a34f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:24.596579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fd1c6-cddc-494d-b84b-5dc9c361a34f admin admin] PATH_INFO: `/v3/projects/9340450b91db44459cc62a2276957230/users/525f7f33ec554dffbdad5d372805d0b2/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:24.598587 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fd1c6-cddc-494d-b84b-5dc9c361a34f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9340450b91db44459cc62a2276957230, user_id=525f7f33ec554dffbdad5d372805d0b2, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:24.601315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fd1c6-cddc-494d-b84b-5dc9c361a34f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:24.612971 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1337/2653] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 03:59:24 2022] PUT /identity/v3/projects/9340450b91db44459cc62a2276957230/users/525f7f33ec554dffbdad5d372805d0b2/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 03:59:28.791692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377cedf-e505-4ab3-bd8d-e99826db1dc6 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:28.792291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377cedf-e505-4ab3-bd8d-e99826db1dc6 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:28.792483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377cedf-e505-4ab3-bd8d-e99826db1dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:28.820149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-a377cedf-e505-4ab3-bd8d-e99826db1dc6 None None] MFA Rules not processed for user `525f7f33ec554dffbdad5d372805d0b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:59:28.821512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-a377cedf-e505-4ab3-bd8d-e99826db1dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:28.853861 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:59:28.898358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1317/2654] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:59:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:59:28.914663 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2fd1c6-cddc-494d-b84b-5dc9c361a34f admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:28.916814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1e2fd1c6-cddc-494d-b84b-5dc9c361a34f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:29.004185 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2fd1c6-cddc-494d-b84b-5dc9c361a34f admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:29.009964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a6f7a-a9d6-4d1f-80a4-3b7a5d3b4898 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:29.010630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a6f7a-a9d6-4d1f-80a4-3b7a5d3b4898 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:29.011007 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a6f7a-a9d6-4d1f-80a4-3b7a5d3b4898 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:29.011282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a6f7a-a9d6-4d1f-80a4-3b7a5d3b4898 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:29.067305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a6f7a-a9d6-4d1f-80a4-3b7a5d3b4898 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:29.068964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a6f7a-a9d6-4d1f-80a4-3b7a5d3b4898 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:29.078370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1338/2655] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:59:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:59:30.151566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7b0c3-bd4a-48f7-88e4-074dd53d80f5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:30.152169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7b0c3-bd4a-48f7-88e4-074dd53d80f5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:30.152281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7b0c3-bd4a-48f7-88e4-074dd53d80f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:30.171913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-89c7b0c3-bd4a-48f7-88e4-074dd53d80f5 None None] MFA Rules not processed for user `5991e17e8dfb4a59a4dd1a4da24ff90c`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 03:59:30.173255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-89c7b0c3-bd4a-48f7-88e4-074dd53d80f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:30.180989 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1318/2656] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 03:59:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 03:59:39.074144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:INSERT=2,keystone:SELECT=63 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:59:40.166296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=18 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:59:45.372445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a6f7a-a9d6-4d1f-80a4-3b7a5d3b4898 service nova] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:45.373933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-519a6f7a-a9d6-4d1f-80a4-3b7a5d3b4898 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:45.436763 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a6f7a-a9d6-4d1f-80a4-3b7a5d3b4898 service nova] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:45.442738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf8454-acb7-4d73-948f-60a61c0549cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:45.443679 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf8454-acb7-4d73-948f-60a61c0549cd service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:45.444093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf8454-acb7-4d73-948f-60a61c0549cd service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:45.444453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf8454-acb7-4d73-948f-60a61c0549cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:45.446733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4abf8454-acb7-4d73-948f-60a61c0549cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:45.461084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf8454-acb7-4d73-948f-60a61c0549cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:45.462525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf8454-acb7-4d73-948f-60a61c0549cd service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:45.473533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1339/2657] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:59:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:59:55.468609 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=26 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 03:59:56.619720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7b0c3-bd4a-48f7-88e4-074dd53d80f5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:56.621386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-89c7b0c3-bd4a-48f7-88e4-074dd53d80f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:56.635486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7b0c3-bd4a-48f7-88e4-074dd53d80f5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:56.639897 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21d154-76fe-426a-8ac5-84acaa598930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:56.640634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21d154-76fe-426a-8ac5-84acaa598930 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:56.640940 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21d154-76fe-426a-8ac5-84acaa598930 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:56.641207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21d154-76fe-426a-8ac5-84acaa598930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:56.643435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3c21d154-76fe-426a-8ac5-84acaa598930 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:56.734363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21d154-76fe-426a-8ac5-84acaa598930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:56.737159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21d154-76fe-426a-8ac5-84acaa598930 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:56.759820 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1319/2658] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:59:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 03:59:58.292329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf8454-acb7-4d73-948f-60a61c0549cd service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 03:59:58.293928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-4abf8454-acb7-4d73-948f-60a61c0549cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:58.358897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf8454-acb7-4d73-948f-60a61c0549cd service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 03:59:58.363500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a88d2-5a25-48c2-bdc9-739ecae79fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 03:59:58.364384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a88d2-5a25-48c2-bdc9-739ecae79fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 03:59:58.364598 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a88d2-5a25-48c2-bdc9-739ecae79fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 03:59:58.365611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a88d2-5a25-48c2-bdc9-739ecae79fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 03:59:58.368132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0a6a88d2-5a25-48c2-bdc9-739ecae79fc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 03:59:58.379067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a88d2-5a25-48c2-bdc9-739ecae79fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 03:59:58.380558 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a88d2-5a25-48c2-bdc9-739ecae79fc1 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 03:59:58.389641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1340/2659] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 03:59:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:06.756827 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=41 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:07.539541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21d154-76fe-426a-8ac5-84acaa598930 service cinder] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:07.549252 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21d154-76fe-426a-8ac5-84acaa598930 service cinder] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:07.552890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbcfaa-2ea9-4a8e-810f-1f486c9479fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:07.553590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbcfaa-2ea9-4a8e-810f-1f486c9479fa service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:07.553880 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbcfaa-2ea9-4a8e-810f-1f486c9479fa service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:07.554162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbcfaa-2ea9-4a8e-810f-1f486c9479fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:07.555934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-13cbcfaa-2ea9-4a8e-810f-1f486c9479fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:07.620987 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbcfaa-2ea9-4a8e-810f-1f486c9479fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:07.622366 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbcfaa-2ea9-4a8e-810f-1f486c9479fa service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:07.633322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1320/2660] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:08.385712 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=26 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:09.877593 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a88d2-5a25-48c2-bdc9-739ecae79fc1 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:09.886508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a88d2-5a25-48c2-bdc9-739ecae79fc1 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:09.890679 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5ebbf-4605-44df-8ec6-bfb8a999751a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:09.891334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5ebbf-4605-44df-8ec6-bfb8a999751a service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:09.891624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5ebbf-4605-44df-8ec6-bfb8a999751a service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:09.892454 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5ebbf-4605-44df-8ec6-bfb8a999751a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:09.895328 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-fdf5ebbf-4605-44df-8ec6-bfb8a999751a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:09.909450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5ebbf-4605-44df-8ec6-bfb8a999751a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:09.910903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5ebbf-4605-44df-8ec6-bfb8a999751a service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:09.921728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1341/2661] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:11.024967 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbcfaa-2ea9-4a8e-810f-1f486c9479fa service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:11.026534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-13cbcfaa-2ea9-4a8e-810f-1f486c9479fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:11.041845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbcfaa-2ea9-4a8e-810f-1f486c9479fa service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:11.048622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300335fe-cf01-49f7-ae7d-b504cbfb1573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:11.050049 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300335fe-cf01-49f7-ae7d-b504cbfb1573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:11.051176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300335fe-cf01-49f7-ae7d-b504cbfb1573 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:11.051637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300335fe-cf01-49f7-ae7d-b504cbfb1573 admin admin] PATH_INFO: `/v3/users/bf67549bf7e443c6b77eaf71fb6b709c` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:11.053335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300335fe-cf01-49f7-ae7d-b504cbfb1573 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf67549bf7e443c6b77eaf71fb6b709c)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:11.055519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300335fe-cf01-49f7-ae7d-b504cbfb1573 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:11.145020 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-300335fe-cf01-49f7-ae7d-b504cbfb1573 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf67549bf7e443c6b77eaf71fb6b709c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:11.169190 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-300335fe-cf01-49f7-ae7d-b504cbfb1573 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf67549bf7e443c6b77eaf71fb6b709c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:11.170211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-300335fe-cf01-49f7-ae7d-b504cbfb1573 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf67549bf7e443c6b77eaf71fb6b709c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:11.180786 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-300335fe-cf01-49f7-ae7d-b504cbfb1573 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf67549bf7e443c6b77eaf71fb6b709c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:11.197185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-300335fe-cf01-49f7-ae7d-b504cbfb1573 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf67549bf7e443c6b77eaf71fb6b709c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:11.211776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1321/2662] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:00:11 2022] DELETE /identity/v3/users/bf67549bf7e443c6b77eaf71fb6b709c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:00:11.447443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5ebbf-4605-44df-8ec6-bfb8a999751a service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:11.464170 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5ebbf-4605-44df-8ec6-bfb8a999751a service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:11.533501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:11.534675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:11.534959 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:11.535234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b admin admin] PATH_INFO: `/v3/projects/e9417b8011914e108ae00b5aa583ce50` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:11.536498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9417b8011914e108ae00b5aa583ce50)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:11.538425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:11.594470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9417b8011914e108ae00b5aa583ce50', 'request_id': 'req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:11.602320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9417b8011914e108ae00b5aa583ce50', 'request_id': 'req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:11.609103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9417b8011914e108ae00b5aa583ce50', 'request_id': 'req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:11.620049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b admin admin] The token cache is being invalidate because project e9417b8011914e108ae00b5aa583ce50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:00:11.620494 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:11.621232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ffd699ec-fbc2-4c8b-97a9-e0758aff045b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:11.622829 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1342/2663] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:00:11 2022] DELETE /identity/v3/projects/e9417b8011914e108ae00b5aa583ce50 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:00:11.628186 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aea9c4-73d0-461c-889f-3a6fefab0f01 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:11.628502 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aea9c4-73d0-461c-889f-3a6fefab0f01 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:11.628790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aea9c4-73d0-461c-889f-3a6fefab0f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:11.647446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c5aea9c4-73d0-461c-889f-3a6fefab0f01 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:11.648941 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c5aea9c4-73d0-461c-889f-3a6fefab0f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:11.720127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1322/2664] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 04:00:11.724451 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf94cad-d3a8-482e-8c94-9ed8972e1ec8 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:11.724863 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf94cad-d3a8-482e-8c94-9ed8972e1ec8 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:11.725208 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf94cad-d3a8-482e-8c94-9ed8972e1ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:11.745887 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-0bf94cad-d3a8-482e-8c94-9ed8972e1ec8 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:11.747252 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0bf94cad-d3a8-482e-8c94-9ed8972e1ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:11.755874 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1343/2665] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:00:11.760374 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aea9c4-73d0-461c-889f-3a6fefab0f01 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:11.771351 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aea9c4-73d0-461c-889f-3a6fefab0f01 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:11.776885 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb0ebf-579d-4992-9696-d0d0340f84aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:11.777590 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb0ebf-579d-4992-9696-d0d0340f84aa admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:11.778289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb0ebf-579d-4992-9696-d0d0340f84aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:11.778289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb0ebf-579d-4992-9696-d0d0340f84aa admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:11.779106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb0ebf-579d-4992-9696-d0d0340f84aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 04:00:11.779414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb0ebf-579d-4992-9696-d0d0340f84aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:11.781576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb0ebf-579d-4992-9696-d0d0340f84aa admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:11.783603 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:11.808356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1323/2666] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 04:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 04:00:11.810108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf94cad-d3a8-482e-8c94-9ed8972e1ec8 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:11.821317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf94cad-d3a8-482e-8c94-9ed8972e1ec8 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:11.826388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa13b1-edf2-4a43-b774-7831dbfb6fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:11.827395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa13b1-edf2-4a43-b774-7831dbfb6fed admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:11.827867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa13b1-edf2-4a43-b774-7831dbfb6fed admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:11.828189 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa13b1-edf2-4a43-b774-7831dbfb6fed admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:11.828908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa13b1-edf2-4a43-b774-7831dbfb6fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:11.830868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa13b1-edf2-4a43-b774-7831dbfb6fed admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:11.878803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1344/2667] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:00:11 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 04:00:11.883028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb0ebf-579d-4992-9696-d0d0340f84aa admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:11.908517 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb0ebf-579d-4992-9696-d0d0340f84aa admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:12.052824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618e207-a14a-4aa9-936d-3b3a8405921b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:12.053645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618e207-a14a-4aa9-936d-3b3a8405921b admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:12.054131 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618e207-a14a-4aa9-936d-3b3a8405921b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:12.054614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618e207-a14a-4aa9-936d-3b3a8405921b admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:12.055520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618e207-a14a-4aa9-936d-3b3a8405921b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:12.057750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618e207-a14a-4aa9-936d-3b3a8405921b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:12.091337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1324/2668] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:00:11 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:00:12.095271 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa13b1-edf2-4a43-b774-7831dbfb6fed admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:12.106772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa13b1-edf2-4a43-b774-7831dbfb6fed admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:12.113146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee568922-1233-4dcd-8182-ea15c68009e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:12.114132 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568922-1233-4dcd-8182-ea15c68009e8 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:12.114987 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568922-1233-4dcd-8182-ea15c68009e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:12.115489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568922-1233-4dcd-8182-ea15c68009e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:12.117286 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee568922-1233-4dcd-8182-ea15c68009e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:12.119824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee568922-1233-4dcd-8182-ea15c68009e8 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:12.140773 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618e207-a14a-4aa9-936d-3b3a8405921b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:12.146589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1345/2669] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:00:12.153116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618e207-a14a-4aa9-936d-3b3a8405921b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:12.159773 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c723ad-9b0e-4a53-8fcb-be572c671a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:12.161484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c723ad-9b0e-4a53-8fcb-be572c671a9a admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:12.162149 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c723ad-9b0e-4a53-8fcb-be572c671a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:12.162637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c723ad-9b0e-4a53-8fcb-be572c671a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:12.163960 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c723ad-9b0e-4a53-8fcb-be572c671a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:12.166251 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c723ad-9b0e-4a53-8fcb-be572c671a9a admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:12.190272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1325/2670] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 04:00:12.192091 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee568922-1233-4dcd-8182-ea15c68009e8 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:12.206800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee568922-1233-4dcd-8182-ea15c68009e8 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:12.212982 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2594cb-2412-4a74-af03-0f9ddf5fc6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:12.214672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2594cb-2412-4a74-af03-0f9ddf5fc6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:12.215238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2594cb-2412-4a74-af03-0f9ddf5fc6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:12.216907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2594cb-2412-4a74-af03-0f9ddf5fc6fe admin admin] PATH_INFO: `/v3/projects/5c29114118814a298f6337842ad29a9b/users/0995483e7e7049bbb639ef3142192abe/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:12.234329 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2594cb-2412-4a74-af03-0f9ddf5fc6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c29114118814a298f6337842ad29a9b, user_id=0995483e7e7049bbb639ef3142192abe, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:12.237403 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2594cb-2412-4a74-af03-0f9ddf5fc6fe admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:12.264855 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1346/2671] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:00:12 2022] PUT /identity/v3/projects/5c29114118814a298f6337842ad29a9b/users/0995483e7e7049bbb639ef3142192abe/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:00:12.266663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707d997-7d75-4c64-aa50-db765c9c0cdb None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:12.266965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707d997-7d75-4c64-aa50-db765c9c0cdb None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:12.267245 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707d997-7d75-4c64-aa50-db765c9c0cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:12.286471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2707d997-7d75-4c64-aa50-db765c9c0cdb None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:12.287682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2707d997-7d75-4c64-aa50-db765c9c0cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:12.360593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1326/2672] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:00:12.372944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2594cb-2412-4a74-af03-0f9ddf5fc6fe admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:12.374867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3c2594cb-2412-4a74-af03-0f9ddf5fc6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:12.395871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:12.442402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2594cb-2412-4a74-af03-0f9ddf5fc6fe admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:12.446249 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52114a-bfb9-47f8-8bc9-2b6ea5d68d35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:12.446904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52114a-bfb9-47f8-8bc9-2b6ea5d68d35 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:12.447188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52114a-bfb9-47f8-8bc9-2b6ea5d68d35 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:12.447459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52114a-bfb9-47f8-8bc9-2b6ea5d68d35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:12.458238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52114a-bfb9-47f8-8bc9-2b6ea5d68d35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:12.460038 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52114a-bfb9-47f8-8bc9-2b6ea5d68d35 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:12.473810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1347/2673] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:13.715472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707d997-7d75-4c64-aa50-db765c9c0cdb None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:13.718407 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2707d997-7d75-4c64-aa50-db765c9c0cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:13.763387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707d997-7d75-4c64-aa50-db765c9c0cdb None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:13.776115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a33e86-ff36-43a0-8cce-8859dcada1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:13.777229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33e86-ff36-43a0-8cce-8859dcada1ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:13.777521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33e86-ff36-43a0-8cce-8859dcada1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:13.777801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33e86-ff36-43a0-8cce-8859dcada1ca admin admin] PATH_INFO: `/v3/users/b10d2c10313c4be2a8c11cff1eed8852` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:13.782609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a33e86-ff36-43a0-8cce-8859dcada1ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b10d2c10313c4be2a8c11cff1eed8852)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:13.784711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a33e86-ff36-43a0-8cce-8859dcada1ca admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:13.860277 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-63a33e86-ff36-43a0-8cce-8859dcada1ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b10d2c10313c4be2a8c11cff1eed8852'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:13.875609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-63a33e86-ff36-43a0-8cce-8859dcada1ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b10d2c10313c4be2a8c11cff1eed8852'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:13.876473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-63a33e86-ff36-43a0-8cce-8859dcada1ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b10d2c10313c4be2a8c11cff1eed8852'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:13.890216 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-63a33e86-ff36-43a0-8cce-8859dcada1ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b10d2c10313c4be2a8c11cff1eed8852'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:13.896740 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:13.913111 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-63a33e86-ff36-43a0-8cce-8859dcada1ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b10d2c10313c4be2a8c11cff1eed8852'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:13.952127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1327/2674] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:00:13 2022] DELETE /identity/v3/users/b10d2c10313c4be2a8c11cff1eed8852 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 04:00:14.326063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52114a-bfb9-47f8-8bc9-2b6ea5d68d35 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:14.337252 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52114a-bfb9-47f8-8bc9-2b6ea5d68d35 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:14.409304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:14.410186 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:14.410496 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:14.410768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e admin admin] PATH_INFO: `/v3/projects/2ceaecc3b7fc4fed9dce1e3ab1014ed2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:14.411965 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ceaecc3b7fc4fed9dce1e3ab1014ed2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:14.413906 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:14.467404 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ceaecc3b7fc4fed9dce1e3ab1014ed2', 'request_id': 'req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:14.477419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ceaecc3b7fc4fed9dce1e3ab1014ed2', 'request_id': 'req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:14.486751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ceaecc3b7fc4fed9dce1e3ab1014ed2', 'request_id': 'req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:14.499380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e admin admin] The token cache is being invalidate because project 2ceaecc3b7fc4fed9dce1e3ab1014ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:00:14.504385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:14.505317 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-e4401c4a-a28b-4cbc-bcd3-2e8f058eb01e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:14.507291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1348/2675] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:00:14 2022] DELETE /identity/v3/projects/2ceaecc3b7fc4fed9dce1e3ab1014ed2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 04:00:14.515192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1534c85-b394-4d77-bc3d-4ee5351fead8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:14.515783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1534c85-b394-4d77-bc3d-4ee5351fead8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:14.516235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1534c85-b394-4d77-bc3d-4ee5351fead8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:14.553202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b1534c85-b394-4d77-bc3d-4ee5351fead8 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:14.554463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b1534c85-b394-4d77-bc3d-4ee5351fead8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:14.641568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1328/2676] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 04:00:14.658657 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4e27d-2c78-483b-9aa5-483d3d5a8670 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:14.659008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4e27d-2c78-483b-9aa5-483d3d5a8670 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:14.659302 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4e27d-2c78-483b-9aa5-483d3d5a8670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:14.686048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-53d4e27d-2c78-483b-9aa5-483d3d5a8670 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:14.687483 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-53d4e27d-2c78-483b-9aa5-483d3d5a8670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:14.695492 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1349/2677] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:00:14.699829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1534c85-b394-4d77-bc3d-4ee5351fead8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:14.709461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1534c85-b394-4d77-bc3d-4ee5351fead8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:14.714104 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05af18-e4a0-452c-aca9-92e34b5032c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:14.714774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05af18-e4a0-452c-aca9-92e34b5032c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:14.715057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05af18-e4a0-452c-aca9-92e34b5032c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:14.715330 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05af18-e4a0-452c-aca9-92e34b5032c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:14.716091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05af18-e4a0-452c-aca9-92e34b5032c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 04:00:14.716693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05af18-e4a0-452c-aca9-92e34b5032c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:14.718077 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05af18-e4a0-452c-aca9-92e34b5032c5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:14.733196 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1329/2678] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 04:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 04:00:14.734425 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4e27d-2c78-483b-9aa5-483d3d5a8670 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:14.743682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4e27d-2c78-483b-9aa5-483d3d5a8670 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:14.748363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0b20b-487a-4fb9-82d1-c6f2063ca0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:14.749086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0b20b-487a-4fb9-82d1-c6f2063ca0bd admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:14.749459 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0b20b-487a-4fb9-82d1-c6f2063ca0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:14.749853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0b20b-487a-4fb9-82d1-c6f2063ca0bd admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:14.750711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0b20b-487a-4fb9-82d1-c6f2063ca0bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:14.753720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0b20b-487a-4fb9-82d1-c6f2063ca0bd admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:14.790714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1350/2679] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:00:14 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:00:14.792099 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05af18-e4a0-452c-aca9-92e34b5032c5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:14.802988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05af18-e4a0-452c-aca9-92e34b5032c5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:14.870774 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a48ce8-ae84-4fef-a7d0-04a9e032dc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:14.871466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a48ce8-ae84-4fef-a7d0-04a9e032dc35 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:14.871751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a48ce8-ae84-4fef-a7d0-04a9e032dc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:14.872058 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a48ce8-ae84-4fef-a7d0-04a9e032dc35 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:14.872760 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a48ce8-ae84-4fef-a7d0-04a9e032dc35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:14.874856 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a48ce8-ae84-4fef-a7d0-04a9e032dc35 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:14.899360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1330/2680] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:00:14 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:00:14.900718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0b20b-487a-4fb9-82d1-c6f2063ca0bd admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:14.911881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0b20b-487a-4fb9-82d1-c6f2063ca0bd admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:14.916899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3090ba-5287-432f-ad50-88fd546a53c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:14.917574 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3090ba-5287-432f-ad50-88fd546a53c2 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:14.917856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3090ba-5287-432f-ad50-88fd546a53c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:14.918126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3090ba-5287-432f-ad50-88fd546a53c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:14.918853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3090ba-5287-432f-ad50-88fd546a53c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:14.920518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3090ba-5287-432f-ad50-88fd546a53c2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:14.931368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1351/2681] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:00:14.949283 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a48ce8-ae84-4fef-a7d0-04a9e032dc35 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:14.962290 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a48ce8-ae84-4fef-a7d0-04a9e032dc35 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:14.971557 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6581dd-c93a-4e0d-ae4f-f3b05749f661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:14.972441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6581dd-c93a-4e0d-ae4f-f3b05749f661 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:14.972882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6581dd-c93a-4e0d-ae4f-f3b05749f661 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:14.973304 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6581dd-c93a-4e0d-ae4f-f3b05749f661 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:14.974154 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6581dd-c93a-4e0d-ae4f-f3b05749f661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:14.976408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6581dd-c93a-4e0d-ae4f-f3b05749f661 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:14.989209 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1331/2682] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:00:14.993803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3090ba-5287-432f-ad50-88fd546a53c2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:15.006484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3090ba-5287-432f-ad50-88fd546a53c2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:15.011337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867aeb2-af6f-49e4-9478-6387248a4947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:15.012662 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867aeb2-af6f-49e4-9478-6387248a4947 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:15.013030 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867aeb2-af6f-49e4-9478-6387248a4947 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:15.013367 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867aeb2-af6f-49e4-9478-6387248a4947 admin admin] PATH_INFO: `/v3/projects/6faaef877fb04e7ea46af3a8e6dbb318/users/63e79858394b4594933646f21fc69db4/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:15.016375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:15.035358 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867aeb2-af6f-49e4-9478-6387248a4947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6faaef877fb04e7ea46af3a8e6dbb318, user_id=63e79858394b4594933646f21fc69db4, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:15.038333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867aeb2-af6f-49e4-9478-6387248a4947 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:15.054638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1352/2683] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:00:14 2022] PUT /identity/v3/projects/6faaef877fb04e7ea46af3a8e6dbb318/users/63e79858394b4594933646f21fc69db4/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 04:00:15.067516 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefcc37-07bd-4c54-95a7-b98c811f1485 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:15.067638 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefcc37-07bd-4c54-95a7-b98c811f1485 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:15.067851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefcc37-07bd-4c54-95a7-b98c811f1485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:15.113200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1aefcc37-07bd-4c54-95a7-b98c811f1485 None None] MFA Rules not processed for user `63e79858394b4594933646f21fc69db4`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:15.114580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1aefcc37-07bd-4c54-95a7-b98c811f1485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:15.201042 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:15.253148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1332/2684] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:00:15.254327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867aeb2-af6f-49e4-9478-6387248a4947 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:15.402624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867aeb2-af6f-49e4-9478-6387248a4947 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:15.417102 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b1ac4-5e07-4cf6-8b05-c060ebc8153c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:15.417792 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b1ac4-5e07-4cf6-8b05-c060ebc8153c admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:15.418079 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b1ac4-5e07-4cf6-8b05-c060ebc8153c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:15.418349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b1ac4-5e07-4cf6-8b05-c060ebc8153c admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:15.419055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b1ac4-5e07-4cf6-8b05-c060ebc8153c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:15.421505 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b1ac4-5e07-4cf6-8b05-c060ebc8153c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:15.462581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1353/2685] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:00:15 2022] POST /identity/v3/projects => generated 377 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:00:15.466800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefcc37-07bd-4c54-95a7-b98c811f1485 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:15.478198 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefcc37-07bd-4c54-95a7-b98c811f1485 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:15.540237 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56efe5-8b35-4fce-b503-df3a162c97d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:15.541035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56efe5-8b35-4fce-b503-df3a162c97d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:15.541333 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56efe5-8b35-4fce-b503-df3a162c97d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:15.541611 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea56efe5-8b35-4fce-b503-df3a162c97d5 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:15.542322 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56efe5-8b35-4fce-b503-df3a162c97d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:15.544339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea56efe5-8b35-4fce-b503-df3a162c97d5 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:15.567334 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1333/2686] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:00:15 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:00:15.571563 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b1ac4-5e07-4cf6-8b05-c060ebc8153c admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:15.583299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b1ac4-5e07-4cf6-8b05-c060ebc8153c admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:15.588858 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f5fa-d92f-41f3-9122-ad894dc14ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:15.589732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f5fa-d92f-41f3-9122-ad894dc14ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:15.590123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f5fa-d92f-41f3-9122-ad894dc14ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:15.590499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f5fa-d92f-41f3-9122-ad894dc14ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:15.591333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2f5fa-d92f-41f3-9122-ad894dc14ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:15.593691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2f5fa-d92f-41f3-9122-ad894dc14ae9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:15.605843 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1354/2687] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:00:15.610427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56efe5-8b35-4fce-b503-df3a162c97d5 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:15.621542 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea56efe5-8b35-4fce-b503-df3a162c97d5 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:15.626551 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831e8e4-c771-4eeb-baec-1bc5bf280e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:15.627481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831e8e4-c771-4eeb-baec-1bc5bf280e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:15.627765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831e8e4-c771-4eeb-baec-1bc5bf280e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:15.628073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831e8e4-c771-4eeb-baec-1bc5bf280e5d admin admin] PATH_INFO: `/v3/projects/ed27fd8265054594a785b4b422768d4e/users/eac2e8f6c77e447cb4ba9eba4b264aab/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:15.645955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831e8e4-c771-4eeb-baec-1bc5bf280e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed27fd8265054594a785b4b422768d4e, user_id=eac2e8f6c77e447cb4ba9eba4b264aab, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:15.648338 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831e8e4-c771-4eeb-baec-1bc5bf280e5d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:15.665655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1334/2688] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:00:15 2022] PUT /identity/v3/projects/ed27fd8265054594a785b4b422768d4e/users/eac2e8f6c77e447cb4ba9eba4b264aab/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:00:15.669743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f5fa-d92f-41f3-9122-ad894dc14ae9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:15.681507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f5fa-d92f-41f3-9122-ad894dc14ae9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:15.749397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd59f5e-f593-4fb0-9c20-3ac4e152eff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:15.750484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd59f5e-f593-4fb0-9c20-3ac4e152eff6 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:15.750484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd59f5e-f593-4fb0-9c20-3ac4e152eff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:15.750775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd59f5e-f593-4fb0-9c20-3ac4e152eff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:15.751463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd59f5e-f593-4fb0-9c20-3ac4e152eff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:15.753092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd59f5e-f593-4fb0-9c20-3ac4e152eff6 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:15.764251 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1355/2689] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 04:00:15.768332 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831e8e4-c771-4eeb-baec-1bc5bf280e5d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:15.782645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831e8e4-c771-4eeb-baec-1bc5bf280e5d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:15.789535 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd1dc2-6781-4753-aab1-69399c94c71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:15.791720 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd1dc2-6781-4753-aab1-69399c94c71f admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:15.792373 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd1dc2-6781-4753-aab1-69399c94c71f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:15.793031 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd1dc2-6781-4753-aab1-69399c94c71f admin admin] PATH_INFO: `/v3/projects/ed27fd8265054594a785b4b422768d4e/users/eac2e8f6c77e447cb4ba9eba4b264aab/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:15.795464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd1dc2-6781-4753-aab1-69399c94c71f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed27fd8265054594a785b4b422768d4e, user_id=eac2e8f6c77e447cb4ba9eba4b264aab, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:15.798787 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd1dc2-6781-4753-aab1-69399c94c71f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:15.823758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a1e52-d60e-43cc-9d96-ad51282a6218 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:15.824378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a1e52-d60e-43cc-9d96-ad51282a6218 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:15.824728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a1e52-d60e-43cc-9d96-ad51282a6218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:15.825311 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1335/2690] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:00:15 2022] PUT /identity/v3/projects/ed27fd8265054594a785b4b422768d4e/users/eac2e8f6c77e447cb4ba9eba4b264aab/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:00:15.859354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-c92a1e52-d60e-43cc-9d96-ad51282a6218 None None] MFA Rules not processed for user `eac2e8f6c77e447cb4ba9eba4b264aab`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:15.860702 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c92a1e52-d60e-43cc-9d96-ad51282a6218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:15.977258 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:16.060988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1356/2691] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:00:16.071830 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd1dc2-6781-4753-aab1-69399c94c71f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:16.075094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-95cd1dc2-6781-4753-aab1-69399c94c71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:16.148333 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd1dc2-6781-4753-aab1-69399c94c71f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:16.155069 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3835661c-118d-4029-bb75-63bed897e0c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:16.155783 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835661c-118d-4029-bb75-63bed897e0c6 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:16.156188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835661c-118d-4029-bb75-63bed897e0c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:16.156808 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3835661c-118d-4029-bb75-63bed897e0c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:16.213376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:16.227901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835661c-118d-4029-bb75-63bed897e0c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:16.229550 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3835661c-118d-4029-bb75-63bed897e0c6 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:16.242184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1336/2692] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:16.924948 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a1e52-d60e-43cc-9d96-ad51282a6218 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:16.926787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c92a1e52-d60e-43cc-9d96-ad51282a6218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:16.993570 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a1e52-d60e-43cc-9d96-ad51282a6218 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:16.997529 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb3147-7317-43d1-bc16-5774364a91b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:16.998455 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb3147-7317-43d1-bc16-5774364a91b5 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:16.998928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb3147-7317-43d1-bc16-5774364a91b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:16.999357 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb3147-7317-43d1-bc16-5774364a91b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:17.004919 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5afb3147-7317-43d1-bc16-5774364a91b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:17.078521 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb3147-7317-43d1-bc16-5774364a91b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:17.080825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb3147-7317-43d1-bc16-5774364a91b5 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:17.091722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1357/2693] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:17.369751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3dd9d8-2721-4755-8472-d7399db2d1f4 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:17.370213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3dd9d8-2721-4755-8472-d7399db2d1f4 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:17.370310 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3dd9d8-2721-4755-8472-d7399db2d1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:17.427300 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3d3dd9d8-2721-4755-8472-d7399db2d1f4 None None] MFA Rules not processed for user `0995483e7e7049bbb639ef3142192abe`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:17.428660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3d3dd9d8-2721-4755-8472-d7399db2d1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:17.544043 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1337/2694] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 04:00:17.559252 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb3147-7317-43d1-bc16-5774364a91b5 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:17.561042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5afb3147-7317-43d1-bc16-5774364a91b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:17.654021 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb3147-7317-43d1-bc16-5774364a91b5 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:17.660355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a1eb5-605f-4ad6-a525-6f411449e278 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:17.661031 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a1eb5-605f-4ad6-a525-6f411449e278 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:17.661324 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a1eb5-605f-4ad6-a525-6f411449e278 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:17.661607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a1eb5-605f-4ad6-a525-6f411449e278 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:17.670607 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a1eb5-605f-4ad6-a525-6f411449e278 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:17.672833 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a1eb5-605f-4ad6-a525-6f411449e278 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:17.686500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1358/2695] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:19.346838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709f12e-0352-40e5-8710-169f9cb98429 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:19.347388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709f12e-0352-40e5-8710-169f9cb98429 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:19.347388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709f12e-0352-40e5-8710-169f9cb98429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:19.377694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-7709f12e-0352-40e5-8710-169f9cb98429 None None] MFA Rules not processed for user `0995483e7e7049bbb639ef3142192abe`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:19.378929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7709f12e-0352-40e5-8710-169f9cb98429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:19.395425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1338/2696] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:00:19.716151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a1eb5-605f-4ad6-a525-6f411449e278 service nova] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:19.728397 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a1eb5-605f-4ad6-a525-6f411449e278 service nova] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:19.732979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b3320-ea53-4642-a112-cee1be45719d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:19.733733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b3320-ea53-4642-a112-cee1be45719d service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:19.734150 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b3320-ea53-4642-a112-cee1be45719d service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:19.734490 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b3320-ea53-4642-a112-cee1be45719d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:19.736601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-679b3320-ea53-4642-a112-cee1be45719d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:19.749443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b3320-ea53-4642-a112-cee1be45719d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:19.751074 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b3320-ea53-4642-a112-cee1be45719d service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:19.756573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=100 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:19.764196 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1359/2697] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:20.170044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709f12e-0352-40e5-8710-169f9cb98429 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:20.172111 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7709f12e-0352-40e5-8710-169f9cb98429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:20.263694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709f12e-0352-40e5-8710-169f9cb98429 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:20.270043 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d0a80-24d4-4bf0-87e2-80261ca341e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:20.270781 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d0a80-24d4-4bf0-87e2-80261ca341e1 service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:20.271127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d0a80-24d4-4bf0-87e2-80261ca341e1 service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:20.271467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d0a80-24d4-4bf0-87e2-80261ca341e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:20.273846 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7e1d0a80-24d4-4bf0-87e2-80261ca341e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:20.288028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d0a80-24d4-4bf0-87e2-80261ca341e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:20.289875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d0a80-24d4-4bf0-87e2-80261ca341e1 service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:20.301471 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1339/2698] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:23.154327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b3320-ea53-4642-a112-cee1be45719d service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:23.165549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b3320-ea53-4642-a112-cee1be45719d service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:23.169176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919ef17-337e-4eba-97e1-fbda2436b993 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:23.169836 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919ef17-337e-4eba-97e1-fbda2436b993 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:23.170125 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919ef17-337e-4eba-97e1-fbda2436b993 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:23.170399 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919ef17-337e-4eba-97e1-fbda2436b993 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:23.180422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919ef17-337e-4eba-97e1-fbda2436b993 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:23.181886 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919ef17-337e-4eba-97e1-fbda2436b993 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:23.192098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1360/2699] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:30.296542 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=73 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:33.187586 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=6 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:46.165501 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d0a80-24d4-4bf0-87e2-80261ca341e1 service placement] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:46.177946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d0a80-24d4-4bf0-87e2-80261ca341e1 service placement] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:46.184467 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255354a4-40e5-4487-9d79-2aa5bf157f0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:46.185147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255354a4-40e5-4487-9d79-2aa5bf157f0b service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:46.185410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255354a4-40e5-4487-9d79-2aa5bf157f0b service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:46.185697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255354a4-40e5-4487-9d79-2aa5bf157f0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:46.187506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-255354a4-40e5-4487-9d79-2aa5bf157f0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:46.200278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255354a4-40e5-4487-9d79-2aa5bf157f0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:46.201877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255354a4-40e5-4487-9d79-2aa5bf157f0b service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:46.212561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1340/2700] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:54.456170 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919ef17-337e-4eba-97e1-fbda2436b993 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:54.529898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919ef17-337e-4eba-97e1-fbda2436b993 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:54.535595 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:54.536501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:54.536785 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:54.537062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] PATH_INFO: `/v3/users/63e79858394b4594933646f21fc69db4` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:54.538375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] RBAC: Authorizing `identity:delete_user(user_id=63e79858394b4594933646f21fc69db4)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:54.540433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:54.591794 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63e79858394b4594933646f21fc69db4'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:54.592501 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63e79858394b4594933646f21fc69db4'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:54.600300 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63e79858394b4594933646f21fc69db4'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:54.607246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63e79858394b4594933646f21fc69db4'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:54.619322 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63e79858394b4594933646f21fc69db4'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:54.625289 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1361/2701] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:00:54 2022] DELETE /identity/v3/users/63e79858394b4594933646f21fc69db4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:00:54.629881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b209c3-75b5-4dad-bcd4-6c8cedf55a81 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:54.630384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b209c3-75b5-4dad-bcd4-6c8cedf55a81 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:54.630505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b209c3-75b5-4dad-bcd4-6c8cedf55a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:54.648991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-10b209c3-75b5-4dad-bcd4-6c8cedf55a81 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:54.650236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-10b209c3-75b5-4dad-bcd4-6c8cedf55a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:54.712260 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1341/2702] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:00:54.722427 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:54.724109 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:54.788403 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9494e44-48d0-4385-a018-a4a3c0b9a66e admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:54.794696 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5147b2-3f8a-4669-8efb-dcf481a64d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:54.795658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5147b2-3f8a-4669-8efb-dcf481a64d55 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:54.796065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5147b2-3f8a-4669-8efb-dcf481a64d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:54.796431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5147b2-3f8a-4669-8efb-dcf481a64d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:54.807189 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5147b2-3f8a-4669-8efb-dcf481a64d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:54.808732 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5147b2-3f8a-4669-8efb-dcf481a64d55 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:54.819662 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1362/2703] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:55.108663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b209c3-75b5-4dad-bcd4-6c8cedf55a81 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:55.110199 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-10b209c3-75b5-4dad-bcd4-6c8cedf55a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:55.122469 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b209c3-75b5-4dad-bcd4-6c8cedf55a81 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:55.127576 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:55.128489 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:55.128767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:55.129034 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] PATH_INFO: `/v3/projects/6faaef877fb04e7ea46af3a8e6dbb318` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:55.131353 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6faaef877fb04e7ea46af3a8e6dbb318)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:55.133450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:55.182056 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6faaef877fb04e7ea46af3a8e6dbb318', 'request_id': 'req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.191387 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6faaef877fb04e7ea46af3a8e6dbb318', 'request_id': 'req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.197481 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6faaef877fb04e7ea46af3a8e6dbb318', 'request_id': 'req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.208520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] The token cache is being invalidate because project 6faaef877fb04e7ea46af3a8e6dbb318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:00:55.208814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.209436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.210988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1342/2704] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:00:55 2022] DELETE /identity/v3/projects/6faaef877fb04e7ea46af3a8e6dbb318 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:00:55.215087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5147b2-3f8a-4669-8efb-dcf481a64d55 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:55.224634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5147b2-3f8a-4669-8efb-dcf481a64d55 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:55.246133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:55.289846 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad49d9-3e27-4e14-8e48-7c8bc7cec63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:55.290728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad49d9-3e27-4e14-8e48-7c8bc7cec63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:55.291011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad49d9-3e27-4e14-8e48-7c8bc7cec63a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:55.291290 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad49d9-3e27-4e14-8e48-7c8bc7cec63a admin admin] PATH_INFO: `/v3/users/eac2e8f6c77e447cb4ba9eba4b264aab` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:55.292537 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad49d9-3e27-4e14-8e48-7c8bc7cec63a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac2e8f6c77e447cb4ba9eba4b264aab)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:55.294499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad49d9-3e27-4e14-8e48-7c8bc7cec63a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:55.343411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a7ad49d9-3e27-4e14-8e48-7c8bc7cec63a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac2e8f6c77e447cb4ba9eba4b264aab'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.344092 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a7ad49d9-3e27-4e14-8e48-7c8bc7cec63a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac2e8f6c77e447cb4ba9eba4b264aab'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.351477 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a7ad49d9-3e27-4e14-8e48-7c8bc7cec63a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac2e8f6c77e447cb4ba9eba4b264aab'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.358348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a7ad49d9-3e27-4e14-8e48-7c8bc7cec63a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac2e8f6c77e447cb4ba9eba4b264aab'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.370143 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a7ad49d9-3e27-4e14-8e48-7c8bc7cec63a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac2e8f6c77e447cb4ba9eba4b264aab'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.376340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1363/2705] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:00:55 2022] DELETE /identity/v3/users/eac2e8f6c77e447cb4ba9eba4b264aab => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:00:55.666139 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:55.667761 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:55.681280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a8de5-e88d-43fc-9d56-3e44ff131d3f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:55.738278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:55.739148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:55.739306 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:55.739658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc admin admin] PATH_INFO: `/v3/projects/ed27fd8265054594a785b4b422768d4e` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:55.740889 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed27fd8265054594a785b4b422768d4e)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:55.742817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:55.754398 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:55.793087 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed27fd8265054594a785b4b422768d4e', 'request_id': 'req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.802319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed27fd8265054594a785b4b422768d4e', 'request_id': 'req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.808262 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed27fd8265054594a785b4b422768d4e', 'request_id': 'req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.820047 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc admin admin] The token cache is being invalidate because project ed27fd8265054594a785b4b422768d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:00:55.820361 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.821114 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-0b8ed2a0-c674-44ca-975e-ab3022aa2cbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:55.822659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1343/2706] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:00:55 2022] DELETE /identity/v3/projects/ed27fd8265054594a785b4b422768d4e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:00:55.828991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d1b2f-fe1b-4cfb-b6dc-ca3a5a8b2f3d None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:55.829531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d1b2f-fe1b-4cfb-b6dc-ca3a5a8b2f3d None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:55.829531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d1b2f-fe1b-4cfb-b6dc-ca3a5a8b2f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:55.848911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8c2d1b2f-fe1b-4cfb-b6dc-ca3a5a8b2f3d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:55.850113 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8c2d1b2f-fe1b-4cfb-b6dc-ca3a5a8b2f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:55.909119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1364/2707] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 04:00:55.918225 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57187d-3deb-482d-a88c-1298467ad519 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:55.918519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57187d-3deb-482d-a88c-1298467ad519 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:55.918801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57187d-3deb-482d-a88c-1298467ad519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:55.938622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-da57187d-3deb-482d-a88c-1298467ad519 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:55.939855 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-da57187d-3deb-482d-a88c-1298467ad519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:55.948992 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1344/2708] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:00:55.953298 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d1b2f-fe1b-4cfb-b6dc-ca3a5a8b2f3d None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:55.963347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d1b2f-fe1b-4cfb-b6dc-ca3a5a8b2f3d None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:55.969278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236771e4-9bbf-4ac4-937e-5371bfc6998c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:55.969946 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236771e4-9bbf-4ac4-937e-5371bfc6998c admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:55.970228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236771e4-9bbf-4ac4-937e-5371bfc6998c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:55.970500 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236771e4-9bbf-4ac4-937e-5371bfc6998c admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:55.971229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236771e4-9bbf-4ac4-937e-5371bfc6998c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 04:00:55.971534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236771e4-9bbf-4ac4-937e-5371bfc6998c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:55.973196 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236771e4-9bbf-4ac4-937e-5371bfc6998c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:55.985545 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1365/2709] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 04:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 04:00:55.990169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57187d-3deb-482d-a88c-1298467ad519 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:55.999945 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57187d-3deb-482d-a88c-1298467ad519 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:56.005912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153014a-178c-41ea-a97a-0ccc848111ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:56.006575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153014a-178c-41ea-a97a-0ccc848111ed admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:56.006863 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153014a-178c-41ea-a97a-0ccc848111ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:56.007137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153014a-178c-41ea-a97a-0ccc848111ed admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:56.007838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153014a-178c-41ea-a97a-0ccc848111ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:56.009896 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153014a-178c-41ea-a97a-0ccc848111ed admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:56.039627 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1345/2710] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:00:55 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:00:56.043877 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236771e4-9bbf-4ac4-937e-5371bfc6998c admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:56.054238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236771e4-9bbf-4ac4-937e-5371bfc6998c admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:56.088075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:56.121888 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543fae4-ab2d-4534-b363-0b32b15632ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:56.122558 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543fae4-ab2d-4534-b363-0b32b15632ab admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:56.122843 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543fae4-ab2d-4534-b363-0b32b15632ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:56.123117 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543fae4-ab2d-4534-b363-0b32b15632ab admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:56.123797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543fae4-ab2d-4534-b363-0b32b15632ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:56.125806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543fae4-ab2d-4534-b363-0b32b15632ab admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:56.147446 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1366/2711] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:00:56 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:00:56.151707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153014a-178c-41ea-a97a-0ccc848111ed admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:56.162423 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153014a-178c-41ea-a97a-0ccc848111ed admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:56.167318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e4d95-6fa3-4a41-9abd-fc33e0aa15f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:56.167988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e4d95-6fa3-4a41-9abd-fc33e0aa15f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:56.168329 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e4d95-6fa3-4a41-9abd-fc33e0aa15f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:56.168581 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e4d95-6fa3-4a41-9abd-fc33e0aa15f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:56.169289 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e4d95-6fa3-4a41-9abd-fc33e0aa15f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:56.170901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e4d95-6fa3-4a41-9abd-fc33e0aa15f9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:56.181256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1346/2712] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:00:56.185116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543fae4-ab2d-4534-b363-0b32b15632ab admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:56.196095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543fae4-ab2d-4534-b363-0b32b15632ab admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:56.201146 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f8042-ecea-49d2-a0da-c32187c98b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:56.201806 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f8042-ecea-49d2-a0da-c32187c98b70 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:56.202095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f8042-ecea-49d2-a0da-c32187c98b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:56.202373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f8042-ecea-49d2-a0da-c32187c98b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:56.203105 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f8042-ecea-49d2-a0da-c32187c98b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:56.204775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f8042-ecea-49d2-a0da-c32187c98b70 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:56.215575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1367/2713] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:00:56.219707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e4d95-6fa3-4a41-9abd-fc33e0aa15f9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:56.230456 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e4d95-6fa3-4a41-9abd-fc33e0aa15f9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:56.235566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fbfe1-9445-4357-b332-3c11db676287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:56.236529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fbfe1-9445-4357-b332-3c11db676287 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:56.236819 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fbfe1-9445-4357-b332-3c11db676287 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:56.237123 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fbfe1-9445-4357-b332-3c11db676287 admin admin] PATH_INFO: `/v3/projects/767c21fda78b4e6fa7e8946473f8f373/users/fd123e027df54891bc7b6e18bdad3030/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:56.252402 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fbfe1-9445-4357-b332-3c11db676287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=767c21fda78b4e6fa7e8946473f8f373, user_id=fd123e027df54891bc7b6e18bdad3030, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:56.255044 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fbfe1-9445-4357-b332-3c11db676287 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:56.270881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1347/2714] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:00:56 2022] PUT /identity/v3/projects/767c21fda78b4e6fa7e8946473f8f373/users/fd123e027df54891bc7b6e18bdad3030/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:00:56.281327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b31c76-c651-40b3-8389-17e574705b42 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:56.281617 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b31c76-c651-40b3-8389-17e574705b42 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:56.281892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b31c76-c651-40b3-8389-17e574705b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:56.309094 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-88b31c76-c651-40b3-8389-17e574705b42 None None] MFA Rules not processed for user `fd123e027df54891bc7b6e18bdad3030`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:56.310297 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-88b31c76-c651-40b3-8389-17e574705b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:56.374807 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1368/2715] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:00:56.386626 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fbfe1-9445-4357-b332-3c11db676287 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:56.388666 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6e5fbfe1-9445-4357-b332-3c11db676287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:56.448147 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fbfe1-9445-4357-b332-3c11db676287 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:56.452111 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857fc1c-af92-4c6c-8928-87557f0982ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:56.452758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857fc1c-af92-4c6c-8928-87557f0982ac service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:56.453038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857fc1c-af92-4c6c-8928-87557f0982ac service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:56.453305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857fc1c-af92-4c6c-8928-87557f0982ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:56.462400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8857fc1c-af92-4c6c-8928-87557f0982ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:56.463838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8857fc1c-af92-4c6c-8928-87557f0982ac service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:56.472699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1348/2716] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:56.491346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b31c76-c651-40b3-8389-17e574705b42 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:56.558842 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b31c76-c651-40b3-8389-17e574705b42 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:56.564223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:56.565098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:56.565382 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:56.565658 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] PATH_INFO: `/v3/users/fd123e027df54891bc7b6e18bdad3030` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:56.566896 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd123e027df54891bc7b6e18bdad3030)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:56.568867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:56.610573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:56.617164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd123e027df54891bc7b6e18bdad3030'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:56.618093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd123e027df54891bc7b6e18bdad3030'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:56.625845 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd123e027df54891bc7b6e18bdad3030'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:56.633346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd123e027df54891bc7b6e18bdad3030'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:56.646703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd123e027df54891bc7b6e18bdad3030'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:56.653223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1369/2717] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:00:56 2022] DELETE /identity/v3/users/fd123e027df54891bc7b6e18bdad3030 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:00:56.657934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc787c-2634-4a26-a3b7-cab37787a7df None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:56.658222 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc787c-2634-4a26-a3b7-cab37787a7df None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:56.658852 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc787c-2634-4a26-a3b7-cab37787a7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:56.677998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-eafc787c-2634-4a26-a3b7-cab37787a7df None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:56.679201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-eafc787c-2634-4a26-a3b7-cab37787a7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:56.716094 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:56.743654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1349/2718] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:00:56.753533 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:56.755014 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:56.814725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12af80-755d-434b-93a1-4b2c85c521a1 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:56.818415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a39910-fef8-44c4-97f5-67cc426c5ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:56.819069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a39910-fef8-44c4-97f5-67cc426c5ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:56.819353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a39910-fef8-44c4-97f5-67cc426c5ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:56.819630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a39910-fef8-44c4-97f5-67cc426c5ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:56.829284 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a39910-fef8-44c4-97f5-67cc426c5ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:56.830734 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a39910-fef8-44c4-97f5-67cc426c5ce0 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:56.840583 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1370/2719] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:00:57.136583 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc787c-2634-4a26-a3b7-cab37787a7df None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:57.138145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-eafc787c-2634-4a26-a3b7-cab37787a7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:57.153275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc787c-2634-4a26-a3b7-cab37787a7df None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:57.159991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0e177-d1db-40d1-99a8-f65573185f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:57.160900 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0e177-d1db-40d1-99a8-f65573185f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:57.161274 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0e177-d1db-40d1-99a8-f65573185f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:57.161441 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0e177-d1db-40d1-99a8-f65573185f16 admin admin] PATH_INFO: `/v3/projects/767c21fda78b4e6fa7e8946473f8f373` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:57.162801 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0e177-d1db-40d1-99a8-f65573185f16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=767c21fda78b4e6fa7e8946473f8f373)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:57.164924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0e177-d1db-40d1-99a8-f65573185f16 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:57.213960 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-86d0e177-d1db-40d1-99a8-f65573185f16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '767c21fda78b4e6fa7e8946473f8f373', 'request_id': 'req-86d0e177-d1db-40d1-99a8-f65573185f16'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:57.221893 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-86d0e177-d1db-40d1-99a8-f65573185f16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '767c21fda78b4e6fa7e8946473f8f373', 'request_id': 'req-86d0e177-d1db-40d1-99a8-f65573185f16'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:57.227718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-86d0e177-d1db-40d1-99a8-f65573185f16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '767c21fda78b4e6fa7e8946473f8f373', 'request_id': 'req-86d0e177-d1db-40d1-99a8-f65573185f16'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:57.239239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-86d0e177-d1db-40d1-99a8-f65573185f16 admin admin] The token cache is being invalidate because project 767c21fda78b4e6fa7e8946473f8f373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:00:57.239530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-86d0e177-d1db-40d1-99a8-f65573185f16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:57.240193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-86d0e177-d1db-40d1-99a8-f65573185f16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:00:57.241745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1350/2720] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:00:57 2022] DELETE /identity/v3/projects/767c21fda78b4e6fa7e8946473f8f373 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 04:00:57.246821 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2c928-8bd7-410c-a021-75a1dd9a5eb5 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:57.247109 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2c928-8bd7-410c-a021-75a1dd9a5eb5 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:57.247382 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2c928-8bd7-410c-a021-75a1dd9a5eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:57.265834 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8bb2c928-8bd7-410c-a021-75a1dd9a5eb5 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:57.267039 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8bb2c928-8bd7-410c-a021-75a1dd9a5eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:57.330413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1371/2721] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:00:57.339165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19f466-2af4-4b74-95d9-c7f5ff9e36bc None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:57.339512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19f466-2af4-4b74-95d9-c7f5ff9e36bc None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:57.339887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19f466-2af4-4b74-95d9-c7f5ff9e36bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:57.363079 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9c19f466-2af4-4b74-95d9-c7f5ff9e36bc None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:57.364344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9c19f466-2af4-4b74-95d9-c7f5ff9e36bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:57.374120 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1351/2722] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:00:57.378380 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2c928-8bd7-410c-a021-75a1dd9a5eb5 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:57.389780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2c928-8bd7-410c-a021-75a1dd9a5eb5 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:57.397981 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38be40-4d88-4bc6-b1c5-84d8f3867ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:57.399106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38be40-4d88-4bc6-b1c5-84d8f3867ccc admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:57.399616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38be40-4d88-4bc6-b1c5-84d8f3867ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:57.399986 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38be40-4d88-4bc6-b1c5-84d8f3867ccc admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:57.400750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38be40-4d88-4bc6-b1c5-84d8f3867ccc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 04:00:57.401055 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38be40-4d88-4bc6-b1c5-84d8f3867ccc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:57.402738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38be40-4d88-4bc6-b1c5-84d8f3867ccc admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:57.422873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1372/2723] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 04:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 04:00:57.429418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19f466-2af4-4b74-95d9-c7f5ff9e36bc None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:57.444052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19f466-2af4-4b74-95d9-c7f5ff9e36bc None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:57.450828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02665837-7a05-484c-9186-0e7b281ae788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:57.451485 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02665837-7a05-484c-9186-0e7b281ae788 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:57.451765 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02665837-7a05-484c-9186-0e7b281ae788 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:57.452026 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02665837-7a05-484c-9186-0e7b281ae788 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:57.452756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02665837-7a05-484c-9186-0e7b281ae788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:57.454736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02665837-7a05-484c-9186-0e7b281ae788 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:57.481862 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1352/2724] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:00:57 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:00:57.486731 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38be40-4d88-4bc6-b1c5-84d8f3867ccc admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:57.502625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38be40-4d88-4bc6-b1c5-84d8f3867ccc admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:57.550979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:00:57.565759 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e855824-1070-4313-b510-98491e70e37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:57.566450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e855824-1070-4313-b510-98491e70e37a admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:57.566734 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e855824-1070-4313-b510-98491e70e37a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:57.567013 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e855824-1070-4313-b510-98491e70e37a admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:57.567723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e855824-1070-4313-b510-98491e70e37a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:57.569709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e855824-1070-4313-b510-98491e70e37a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:57.591711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1373/2725] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:00:57 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:00:57.595829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02665837-7a05-484c-9186-0e7b281ae788 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:57.606541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02665837-7a05-484c-9186-0e7b281ae788 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:57.611382 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1790de5-3724-4a25-911b-6966428c1318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:57.612121 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1790de5-3724-4a25-911b-6966428c1318 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:57.612408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1790de5-3724-4a25-911b-6966428c1318 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:57.612682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1790de5-3724-4a25-911b-6966428c1318 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:57.613409 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1790de5-3724-4a25-911b-6966428c1318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:57.615032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1790de5-3724-4a25-911b-6966428c1318 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:57.626316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1353/2726] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:00:57.630725 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e855824-1070-4313-b510-98491e70e37a admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:57.644550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e855824-1070-4313-b510-98491e70e37a admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:57.649596 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49fdab-cb9f-4c3d-9b5b-13a9563bba96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:57.650366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49fdab-cb9f-4c3d-9b5b-13a9563bba96 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:57.650656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49fdab-cb9f-4c3d-9b5b-13a9563bba96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:57.650928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49fdab-cb9f-4c3d-9b5b-13a9563bba96 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:57.651667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49fdab-cb9f-4c3d-9b5b-13a9563bba96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:57.653320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49fdab-cb9f-4c3d-9b5b-13a9563bba96 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:57.664346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1374/2727] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:00:57.669841 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1790de5-3724-4a25-911b-6966428c1318 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:57.680339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1790de5-3724-4a25-911b-6966428c1318 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:57.686803 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4be9b4-348c-4f94-bf31-73a0dcc9d732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:57.688085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4be9b4-348c-4f94-bf31-73a0dcc9d732 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:57.688404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4be9b4-348c-4f94-bf31-73a0dcc9d732 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:57.688660 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4be9b4-348c-4f94-bf31-73a0dcc9d732 admin admin] PATH_INFO: `/v3/projects/4c0a7b2dac194a598025eee16657861a/users/fc84b437a5d642308cae7ae72c1d35c9/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:57.707457 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4be9b4-348c-4f94-bf31-73a0dcc9d732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c0a7b2dac194a598025eee16657861a, user_id=fc84b437a5d642308cae7ae72c1d35c9, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:57.710233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4be9b4-348c-4f94-bf31-73a0dcc9d732 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:57.725929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1354/2728] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:00:57 2022] PUT /identity/v3/projects/4c0a7b2dac194a598025eee16657861a/users/fc84b437a5d642308cae7ae72c1d35c9/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 04:00:57.737420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8895ab-29f2-4801-9097-eb0a88192c13 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:57.737714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8895ab-29f2-4801-9097-eb0a88192c13 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:57.737996 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8895ab-29f2-4801-9097-eb0a88192c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:57.767133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8e8895ab-29f2-4801-9097-eb0a88192c13 None None] MFA Rules not processed for user `fc84b437a5d642308cae7ae72c1d35c9`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:00:57.768378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8e8895ab-29f2-4801-9097-eb0a88192c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:57.836304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1375/2729] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:00:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 04:00:57.849435 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4be9b4-348c-4f94-bf31-73a0dcc9d732 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:00:57.851363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-df4be9b4-348c-4f94-bf31-73a0dcc9d732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:00:57.911558 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4be9b4-348c-4f94-bf31-73a0dcc9d732 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:00:57.915710 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5d381-6381-469c-9d53-09a77293a488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:00:57.916389 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5d381-6381-469c-9d53-09a77293a488 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:00:57.916673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5d381-6381-469c-9d53-09a77293a488 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:00:57.916945 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5d381-6381-469c-9d53-09a77293a488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:00:57.926296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5d381-6381-469c-9d53-09a77293a488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:00:57.927747 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5d381-6381-469c-9d53-09a77293a488 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:00:57.937108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1355/2730] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:00:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:01:01.006409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8895ab-29f2-4801-9097-eb0a88192c13 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:01.072315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8895ab-29f2-4801-9097-eb0a88192c13 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:01.077364 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:01.078237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:01.078522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:01.078799 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] PATH_INFO: `/v3/users/fc84b437a5d642308cae7ae72c1d35c9` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:01.080037 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc84b437a5d642308cae7ae72c1d35c9)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:01.081972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:01.131543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc84b437a5d642308cae7ae72c1d35c9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:01.132249 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc84b437a5d642308cae7ae72c1d35c9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:01.141154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc84b437a5d642308cae7ae72c1d35c9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:01.143321 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:01.150173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc84b437a5d642308cae7ae72c1d35c9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:01.167941 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc84b437a5d642308cae7ae72c1d35c9'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:01.174308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1376/2731] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:01:01 2022] DELETE /identity/v3/users/fc84b437a5d642308cae7ae72c1d35c9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:01:01.179703 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb43be-3512-4b39-abfb-7682151b0ac7 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:01.180274 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb43be-3512-4b39-abfb-7682151b0ac7 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:01.180274 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb43be-3512-4b39-abfb-7682151b0ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:01.187243 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:01.201086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-3efb43be-3512-4b39-abfb-7682151b0ac7 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:01:01.202965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3efb43be-3512-4b39-abfb-7682151b0ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:01.264439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1356/2732] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:01:01.274340 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:01.275823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:01.333401 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2876c-5b10-455f-9289-c48114b23c39 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:01.337508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea6c20-07ab-48f9-b33c-23d50f3a3e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:01.338145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea6c20-07ab-48f9-b33c-23d50f3a3e8f service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:01.338423 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea6c20-07ab-48f9-b33c-23d50f3a3e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:01.338693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea6c20-07ab-48f9-b33c-23d50f3a3e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:01.348684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea6c20-07ab-48f9-b33c-23d50f3a3e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:01.350126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea6c20-07ab-48f9-b33c-23d50f3a3e8f service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:01.359448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1377/2733] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:01:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:01:01.630165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb43be-3512-4b39-abfb-7682151b0ac7 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:01.631735 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-3efb43be-3512-4b39-abfb-7682151b0ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:01.645155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb43be-3512-4b39-abfb-7682151b0ac7 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:01.651496 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc1a39-dfa2-4e1c-bff3-056c025320d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:01.652617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc1a39-dfa2-4e1c-bff3-056c025320d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:01.652970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc1a39-dfa2-4e1c-bff3-056c025320d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:01.653155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc1a39-dfa2-4e1c-bff3-056c025320d6 admin admin] PATH_INFO: `/v3/projects/4c0a7b2dac194a598025eee16657861a` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:01.654568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc1a39-dfa2-4e1c-bff3-056c025320d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c0a7b2dac194a598025eee16657861a)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:01.656528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc1a39-dfa2-4e1c-bff3-056c025320d6 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:01.709956 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-94cc1a39-dfa2-4e1c-bff3-056c025320d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c0a7b2dac194a598025eee16657861a', 'request_id': 'req-94cc1a39-dfa2-4e1c-bff3-056c025320d6'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:01.719137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-94cc1a39-dfa2-4e1c-bff3-056c025320d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c0a7b2dac194a598025eee16657861a', 'request_id': 'req-94cc1a39-dfa2-4e1c-bff3-056c025320d6'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:01.724923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-94cc1a39-dfa2-4e1c-bff3-056c025320d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c0a7b2dac194a598025eee16657861a', 'request_id': 'req-94cc1a39-dfa2-4e1c-bff3-056c025320d6'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:01.736057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-94cc1a39-dfa2-4e1c-bff3-056c025320d6 admin admin] The token cache is being invalidate because project 4c0a7b2dac194a598025eee16657861a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:01:01.736349 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-94cc1a39-dfa2-4e1c-bff3-056c025320d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:01.736986 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-94cc1a39-dfa2-4e1c-bff3-056c025320d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:01.738548 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1357/2734] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:01:01 2022] DELETE /identity/v3/projects/4c0a7b2dac194a598025eee16657861a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:01:01.744112 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99066e-a5e0-4531-8dd2-8385a88d0b3a None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:01.744406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99066e-a5e0-4531-8dd2-8385a88d0b3a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:01.744681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99066e-a5e0-4531-8dd2-8385a88d0b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:01.763196 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-eb99066e-a5e0-4531-8dd2-8385a88d0b3a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:01:01.764453 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-eb99066e-a5e0-4531-8dd2-8385a88d0b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:01.828820 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1378/2735] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:01:01.838269 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc80fc7-70e2-4b5d-b067-edd10c65141f None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:01.838636 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc80fc7-70e2-4b5d-b067-edd10c65141f None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:01.839067 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc80fc7-70e2-4b5d-b067-edd10c65141f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:01.860815 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-9fc80fc7-70e2-4b5d-b067-edd10c65141f None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:01:01.862030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-9fc80fc7-70e2-4b5d-b067-edd10c65141f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:01.873235 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1358/2736] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:01:01.878450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99066e-a5e0-4531-8dd2-8385a88d0b3a None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:01.888916 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99066e-a5e0-4531-8dd2-8385a88d0b3a None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:01.896275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8636e40-c344-4e63-836f-0edc141ecfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:01.896943 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8636e40-c344-4e63-836f-0edc141ecfa5 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:01.897234 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8636e40-c344-4e63-836f-0edc141ecfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:01.897506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8636e40-c344-4e63-836f-0edc141ecfa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:01.898247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8636e40-c344-4e63-836f-0edc141ecfa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 04:01:01.898556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8636e40-c344-4e63-836f-0edc141ecfa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:01.900310 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8636e40-c344-4e63-836f-0edc141ecfa5 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:01.913154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1379/2737] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 04:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 04:01:01.917822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc80fc7-70e2-4b5d-b067-edd10c65141f None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:01.927607 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc80fc7-70e2-4b5d-b067-edd10c65141f None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:01.933218 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9505ff-a995-492b-8a02-24896456dc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:01.933877 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9505ff-a995-492b-8a02-24896456dc07 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:01.934162 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9505ff-a995-492b-8a02-24896456dc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:01.934422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9505ff-a995-492b-8a02-24896456dc07 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:01.935109 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9505ff-a995-492b-8a02-24896456dc07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:01.937079 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9505ff-a995-492b-8a02-24896456dc07 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:01.966254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1359/2738] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:01:01 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:01:01.970512 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8636e40-c344-4e63-836f-0edc141ecfa5 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:01.980897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8636e40-c344-4e63-836f-0edc141ecfa5 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:02.038850 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54763041-4f8c-493d-acc2-f8e8db52a57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:02.039513 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54763041-4f8c-493d-acc2-f8e8db52a57a admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:02.039776 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54763041-4f8c-493d-acc2-f8e8db52a57a admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:02.040060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54763041-4f8c-493d-acc2-f8e8db52a57a admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:02.040789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54763041-4f8c-493d-acc2-f8e8db52a57a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:02.042714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54763041-4f8c-493d-acc2-f8e8db52a57a admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:02.051484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:02.067073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1380/2739] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:01:01 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:01:02.072484 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9505ff-a995-492b-8a02-24896456dc07 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:02.083357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9505ff-a995-492b-8a02-24896456dc07 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:02.089359 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8fc7b-428d-444f-bd35-46d397975ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:02.090382 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8fc7b-428d-444f-bd35-46d397975ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:02.090667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8fc7b-428d-444f-bd35-46d397975ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:02.090943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8fc7b-428d-444f-bd35-46d397975ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:02.091685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8fc7b-428d-444f-bd35-46d397975ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:02.093358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8fc7b-428d-444f-bd35-46d397975ce9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:02.104416 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1360/2740] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:01:02.109009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54763041-4f8c-493d-acc2-f8e8db52a57a admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:02.122345 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54763041-4f8c-493d-acc2-f8e8db52a57a admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:02.127968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67fe5cb-9cbd-47d6-bc91-e82e0c020a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:02.128656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fe5cb-9cbd-47d6-bc91-e82e0c020a9f admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:02.128939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fe5cb-9cbd-47d6-bc91-e82e0c020a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:02.129212 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67fe5cb-9cbd-47d6-bc91-e82e0c020a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:02.129939 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67fe5cb-9cbd-47d6-bc91-e82e0c020a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:02.131548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67fe5cb-9cbd-47d6-bc91-e82e0c020a9f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:02.143566 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1381/2741] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 04:01:02.147668 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8fc7b-428d-444f-bd35-46d397975ce9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:02.158036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8fc7b-428d-444f-bd35-46d397975ce9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:02.164202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a720184-3493-4406-b4a3-fb4073c6cfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:02.165095 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a720184-3493-4406-b4a3-fb4073c6cfb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:02.165377 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a720184-3493-4406-b4a3-fb4073c6cfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:02.165651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a720184-3493-4406-b4a3-fb4073c6cfb2 admin admin] PATH_INFO: `/v3/projects/c452689520aa4baa8d4af3413378ff1b/users/f281858dcb054034a773c2ea64f81a17/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:02.181657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a720184-3493-4406-b4a3-fb4073c6cfb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c452689520aa4baa8d4af3413378ff1b, user_id=f281858dcb054034a773c2ea64f81a17, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:02.184403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a720184-3493-4406-b4a3-fb4073c6cfb2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:02.198672 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1361/2742] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:01:02 2022] PUT /identity/v3/projects/c452689520aa4baa8d4af3413378ff1b/users/f281858dcb054034a773c2ea64f81a17/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 04:01:02.209022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77007c1a-ed40-4f7d-8aa3-a5ab3cb54586 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:02.209314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77007c1a-ed40-4f7d-8aa3-a5ab3cb54586 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:02.209589 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77007c1a-ed40-4f7d-8aa3-a5ab3cb54586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:02.236095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-77007c1a-ed40-4f7d-8aa3-a5ab3cb54586 None None] MFA Rules not processed for user `f281858dcb054034a773c2ea64f81a17`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:01:02.237306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-77007c1a-ed40-4f7d-8aa3-a5ab3cb54586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:02.312277 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1382/2743] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:01:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:01:02.330148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a720184-3493-4406-b4a3-fb4073c6cfb2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:02.332344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1a720184-3493-4406-b4a3-fb4073c6cfb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:02.360838 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:02.418534 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a720184-3493-4406-b4a3-fb4073c6cfb2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:02.422704 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74fbb6-d5e9-4c08-a4d5-8b137f50fbf8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7297224dc73c4358b60f1b096c0fb5bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:02.423358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74fbb6-d5e9-4c08-a4d5-8b137f50fbf8 service glance] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:02.423640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74fbb6-d5e9-4c08-a4d5-8b137f50fbf8 service glance] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:02.423901 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74fbb6-d5e9-4c08-a4d5-8b137f50fbf8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:02.432665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74fbb6-d5e9-4c08-a4d5-8b137f50fbf8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:02.434148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74fbb6-d5e9-4c08-a4d5-8b137f50fbf8 service glance] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:02.443725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1362/2744] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:01:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:01:12.308354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:INSERT=1,keystone:SELECT=39 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:12.439647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=25 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:24.267592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77007c1a-ed40-4f7d-8aa3-a5ab3cb54586 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:24.269335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-77007c1a-ed40-4f7d-8aa3-a5ab3cb54586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:24.325970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77007c1a-ed40-4f7d-8aa3-a5ab3cb54586 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:24.329904 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f15ab4-41a1-48a0-8b90-f18b69c1a114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:24.330551 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f15ab4-41a1-48a0-8b90-f18b69c1a114 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:24.330832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f15ab4-41a1-48a0-8b90-f18b69c1a114 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:24.331109 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f15ab4-41a1-48a0-8b90-f18b69c1a114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:24.332970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f2f15ab4-41a1-48a0-8b90-f18b69c1a114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:24.399406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f15ab4-41a1-48a0-8b90-f18b69c1a114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:24.400887 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f15ab4-41a1-48a0-8b90-f18b69c1a114 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:24.411667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1383/2745] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:01:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:01:31.278290 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74fbb6-d5e9-4c08-a4d5-8b137f50fbf8 service glance] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:31.279974 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2e74fbb6-d5e9-4c08-a4d5-8b137f50fbf8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:31.294845 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74fbb6-d5e9-4c08-a4d5-8b137f50fbf8 service glance] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:31.299656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:31.300572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:31.300861 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:31.301168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] PATH_INFO: `/v3/users/46d5c3f98d924538a34c4b3a57fcac14` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:31.316124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46d5c3f98d924538a34c4b3a57fcac14)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:31.318148 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:31.366009 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46d5c3f98d924538a34c4b3a57fcac14'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.373635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46d5c3f98d924538a34c4b3a57fcac14'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.374334 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46d5c3f98d924538a34c4b3a57fcac14'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.381462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46d5c3f98d924538a34c4b3a57fcac14'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.393663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46d5c3f98d924538a34c4b3a57fcac14'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.400228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1363/2746] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:01:31 2022] DELETE /identity/v3/users/46d5c3f98d924538a34c4b3a57fcac14 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:01:31.579862 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f15ab4-41a1-48a0-8b90-f18b69c1a114 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:31.591755 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f15ab4-41a1-48a0-8b90-f18b69c1a114 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:31.651214 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:31.651990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:31.652302 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:31.652573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] PATH_INFO: `/v3/projects/99d9e85c63234c77889f856bf95288f5` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:31.663373 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99d9e85c63234c77889f856bf95288f5)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:31.665336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:31.713144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:31.716823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99d9e85c63234c77889f856bf95288f5', 'request_id': 'req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.725289 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99d9e85c63234c77889f856bf95288f5', 'request_id': 'req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.732762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99d9e85c63234c77889f856bf95288f5', 'request_id': 'req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.743481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] The token cache is being invalidate because project 99d9e85c63234c77889f856bf95288f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:01:31.743768 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.744418 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.746034 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1384/2747] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:01:31 2022] DELETE /identity/v3/projects/99d9e85c63234c77889f856bf95288f5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:01:31.750201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:31.751669 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:31.765079 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f8fe6-ad58-441f-88f2-260dd6055106 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:31.823159 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:31.824083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:31.824371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:31.824650 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] PATH_INFO: `/v3/users/1aace136bb1e4ba289d41b2328b083e7` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:31.838190 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aace136bb1e4ba289d41b2328b083e7)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:31.840183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:31.887656 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aace136bb1e4ba289d41b2328b083e7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.891356 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:31.897462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aace136bb1e4ba289d41b2328b083e7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.898260 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aace136bb1e4ba289d41b2328b083e7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.905937 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aace136bb1e4ba289d41b2328b083e7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.918706 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aace136bb1e4ba289d41b2328b083e7'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:31.925646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1364/2748] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:01:31 2022] DELETE /identity/v3/users/1aace136bb1e4ba289d41b2328b083e7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:01:32.111638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:32.113301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:32.130415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d80be-3cbd-43d5-b743-9b41cc5e6d57 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:32.191944 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296485e-01d8-4c70-a2fb-d5439306c686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:32.192767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296485e-01d8-4c70-a2fb-d5439306c686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:32.193043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296485e-01d8-4c70-a2fb-d5439306c686 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:32.193320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296485e-01d8-4c70-a2fb-d5439306c686 admin admin] PATH_INFO: `/v3/projects/2abeb9c4daa240b08266c6de955e4bba` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:32.204440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296485e-01d8-4c70-a2fb-d5439306c686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2abeb9c4daa240b08266c6de955e4bba)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:32.206378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296485e-01d8-4c70-a2fb-d5439306c686 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:32.256307 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3296485e-01d8-4c70-a2fb-d5439306c686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2abeb9c4daa240b08266c6de955e4bba', 'request_id': 'req-3296485e-01d8-4c70-a2fb-d5439306c686'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:32.265805 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3296485e-01d8-4c70-a2fb-d5439306c686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2abeb9c4daa240b08266c6de955e4bba', 'request_id': 'req-3296485e-01d8-4c70-a2fb-d5439306c686'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:32.272970 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3296485e-01d8-4c70-a2fb-d5439306c686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2abeb9c4daa240b08266c6de955e4bba', 'request_id': 'req-3296485e-01d8-4c70-a2fb-d5439306c686'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:32.283508 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3296485e-01d8-4c70-a2fb-d5439306c686 admin admin] The token cache is being invalidate because project 2abeb9c4daa240b08266c6de955e4bba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:01:32.283788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3296485e-01d8-4c70-a2fb-d5439306c686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:32.284462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3296485e-01d8-4c70-a2fb-d5439306c686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:32.286395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1385/2749] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:01:32 2022] DELETE /identity/v3/projects/2abeb9c4daa240b08266c6de955e4bba => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:01:41.921798 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=12,keystone:DELETE=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:42.280518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=63,keystone:DELETE=11,keystone:INSERT=2 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:48.578419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:48.580180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:48.594739 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e862fb1-462d-4a90-a2e0-ab1eaf8e4ad9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:48.654805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:48.655677 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:48.655961 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:48.656272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] PATH_INFO: `/v3/users/f281858dcb054034a773c2ea64f81a17` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:48.657524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f281858dcb054034a773c2ea64f81a17)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:48.659470 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:48.714041 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f281858dcb054034a773c2ea64f81a17'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:48.723498 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f281858dcb054034a773c2ea64f81a17'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:48.724344 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f281858dcb054034a773c2ea64f81a17'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:48.731850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f281858dcb054034a773c2ea64f81a17'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:48.744729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f281858dcb054034a773c2ea64f81a17'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:48.750999 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1365/2750] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:01:48 2022] DELETE /identity/v3/users/f281858dcb054034a773c2ea64f81a17 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:01:48.755637 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fa866-e467-4fe2-a464-5ef7807df5dd None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:48.756188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fa866-e467-4fe2-a464-5ef7807df5dd None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:48.756296 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fa866-e467-4fe2-a464-5ef7807df5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:48.774550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-e51fa866-e467-4fe2-a464-5ef7807df5dd None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:01:48.775750 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e51fa866-e467-4fe2-a464-5ef7807df5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:48.840458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1386/2751] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:01:48.850421 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:48.852115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:48.913390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcc549-6870-4d78-a577-0d5732038439 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:48.917759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a24f50-a0b0-441b-9bb4-ad608f3d0400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:48.918417 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a24f50-a0b0-441b-9bb4-ad608f3d0400 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:48.918697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a24f50-a0b0-441b-9bb4-ad608f3d0400 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:48.918982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a24f50-a0b0-441b-9bb4-ad608f3d0400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:48.928741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a24f50-a0b0-441b-9bb4-ad608f3d0400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:48.930183 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a24f50-a0b0-441b-9bb4-ad608f3d0400 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:48.940064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1366/2752] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:01:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:01:49.212583 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fa866-e467-4fe2-a464-5ef7807df5dd None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:49.214103 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e51fa866-e467-4fe2-a464-5ef7807df5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:49.229169 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fa866-e467-4fe2-a464-5ef7807df5dd None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:49.235238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cf2db-8705-40ab-8e4a-2926e9e66952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:49.236158 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cf2db-8705-40ab-8e4a-2926e9e66952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:49.236433 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cf2db-8705-40ab-8e4a-2926e9e66952 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:49.236706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cf2db-8705-40ab-8e4a-2926e9e66952 admin admin] PATH_INFO: `/v3/projects/c452689520aa4baa8d4af3413378ff1b` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:49.238121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cf2db-8705-40ab-8e4a-2926e9e66952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c452689520aa4baa8d4af3413378ff1b)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:49.240240 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cf2db-8705-40ab-8e4a-2926e9e66952 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:49.290115 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-272cf2db-8705-40ab-8e4a-2926e9e66952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c452689520aa4baa8d4af3413378ff1b', 'request_id': 'req-272cf2db-8705-40ab-8e4a-2926e9e66952'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:49.301769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-272cf2db-8705-40ab-8e4a-2926e9e66952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c452689520aa4baa8d4af3413378ff1b', 'request_id': 'req-272cf2db-8705-40ab-8e4a-2926e9e66952'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:49.313257 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-272cf2db-8705-40ab-8e4a-2926e9e66952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c452689520aa4baa8d4af3413378ff1b', 'request_id': 'req-272cf2db-8705-40ab-8e4a-2926e9e66952'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:49.325449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-272cf2db-8705-40ab-8e4a-2926e9e66952 admin admin] The token cache is being invalidate because project c452689520aa4baa8d4af3413378ff1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:01:49.325703 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-272cf2db-8705-40ab-8e4a-2926e9e66952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:49.326438 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-272cf2db-8705-40ab-8e4a-2926e9e66952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:01:49.328313 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1387/2753] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:01:49 2022] DELETE /identity/v3/projects/c452689520aa4baa8d4af3413378ff1b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:01:49.334015 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87084c-9ebf-4f27-b68f-c7761c8bbac1 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:49.334491 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87084c-9ebf-4f27-b68f-c7761c8bbac1 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:49.334923 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87084c-9ebf-4f27-b68f-c7761c8bbac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:49.356524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8d87084c-9ebf-4f27-b68f-c7761c8bbac1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:01:49.357657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8d87084c-9ebf-4f27-b68f-c7761c8bbac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:49.375834 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:49.427718 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1367/2754] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:01:49.429182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a41ba-06e2-45fa-9894-76aacce607ba None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:49.429473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a41ba-06e2-45fa-9894-76aacce607ba None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:49.429752 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a41ba-06e2-45fa-9894-76aacce607ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:49.448316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-140a41ba-06e2-45fa-9894-76aacce607ba None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:01:49.449531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-140a41ba-06e2-45fa-9894-76aacce607ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:49.457450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1388/2755] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 04:01:49.461790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87084c-9ebf-4f27-b68f-c7761c8bbac1 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:49.471708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87084c-9ebf-4f27-b68f-c7761c8bbac1 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:49.476425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d61967-6643-4062-b67e-c153806928d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:49.477115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d61967-6643-4062-b67e-c153806928d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:49.477394 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d61967-6643-4062-b67e-c153806928d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:49.477675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d61967-6643-4062-b67e-c153806928d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:49.478410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d61967-6643-4062-b67e-c153806928d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 04:01:49.478724 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d61967-6643-4062-b67e-c153806928d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:49.480378 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d61967-6643-4062-b67e-c153806928d4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:49.493324 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1368/2756] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 04:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 04:01:49.497484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a41ba-06e2-45fa-9894-76aacce607ba None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:49.507057 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a41ba-06e2-45fa-9894-76aacce607ba None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:49.511672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d7968-30ae-464c-b62a-4fbffd0b6972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:49.512366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d7968-30ae-464c-b62a-4fbffd0b6972 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:49.512668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d7968-30ae-464c-b62a-4fbffd0b6972 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:49.513029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d7968-30ae-464c-b62a-4fbffd0b6972 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:49.513639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d7968-30ae-464c-b62a-4fbffd0b6972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:49.515550 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d7968-30ae-464c-b62a-4fbffd0b6972 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:49.545544 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1389/2757] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:01:49 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:01:49.546869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d61967-6643-4062-b67e-c153806928d4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:49.557547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d61967-6643-4062-b67e-c153806928d4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:49.613004 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c12df-9bc3-43f7-b756-550b9fe75392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:49.613817 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c12df-9bc3-43f7-b756-550b9fe75392 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:49.614073 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c12df-9bc3-43f7-b756-550b9fe75392 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:49.614348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c12df-9bc3-43f7-b756-550b9fe75392 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:49.615039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c12df-9bc3-43f7-b756-550b9fe75392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:49.617061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c12df-9bc3-43f7-b756-550b9fe75392 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:49.636934 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1369/2758] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:01:49 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:01:49.641133 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d7968-30ae-464c-b62a-4fbffd0b6972 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:49.651639 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d7968-30ae-464c-b62a-4fbffd0b6972 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:49.656605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ceadcc-ac85-4d2d-8a0d-8bb9bd0a1191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:49.657282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ceadcc-ac85-4d2d-8a0d-8bb9bd0a1191 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:49.657565 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ceadcc-ac85-4d2d-8a0d-8bb9bd0a1191 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:49.657859 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ceadcc-ac85-4d2d-8a0d-8bb9bd0a1191 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:49.658638 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ceadcc-ac85-4d2d-8a0d-8bb9bd0a1191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:49.660285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ceadcc-ac85-4d2d-8a0d-8bb9bd0a1191 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:49.671892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1390/2759] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:01:49.675945 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c12df-9bc3-43f7-b756-550b9fe75392 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:49.686663 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c12df-9bc3-43f7-b756-550b9fe75392 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:49.692612 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32d104-4f24-4c79-bf46-604afd584102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:49.693277 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32d104-4f24-4c79-bf46-604afd584102 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:49.693558 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32d104-4f24-4c79-bf46-604afd584102 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:49.693833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32d104-4f24-4c79-bf46-604afd584102 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:49.694568 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32d104-4f24-4c79-bf46-604afd584102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:49.696218 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32d104-4f24-4c79-bf46-604afd584102 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:49.706482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1370/2760] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:01:49.710997 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ceadcc-ac85-4d2d-8a0d-8bb9bd0a1191 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:49.721185 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ceadcc-ac85-4d2d-8a0d-8bb9bd0a1191 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:49.725910 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb2d2f-04a7-41ab-b37a-b06b5db78bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:49.726795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb2d2f-04a7-41ab-b37a-b06b5db78bab admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:49.727079 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb2d2f-04a7-41ab-b37a-b06b5db78bab admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:49.727355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb2d2f-04a7-41ab-b37a-b06b5db78bab admin admin] PATH_INFO: `/v3/projects/a2953090fcd9480680957f0f9a16a38d/users/ddbbe75ee31e4725a8de89ab9c234125/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:49.741859 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb2d2f-04a7-41ab-b37a-b06b5db78bab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2953090fcd9480680957f0f9a16a38d, user_id=ddbbe75ee31e4725a8de89ab9c234125, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:49.744580 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb2d2f-04a7-41ab-b37a-b06b5db78bab admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:49.762239 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1391/2761] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:01:49 2022] PUT /identity/v3/projects/a2953090fcd9480680957f0f9a16a38d/users/ddbbe75ee31e4725a8de89ab9c234125/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 04:01:49.772804 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d634a0-2bfb-420c-9084-086e347b0744 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:49.773077 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d634a0-2bfb-420c-9084-086e347b0744 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:49.773361 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d634a0-2bfb-420c-9084-086e347b0744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:49.801427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-03d634a0-2bfb-420c-9084-086e347b0744 None None] MFA Rules not processed for user `ddbbe75ee31e4725a8de89ab9c234125`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:01:49.802630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-03d634a0-2bfb-420c-9084-086e347b0744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:49.869612 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1371/2762] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:01:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:01:49.880448 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb2d2f-04a7-41ab-b37a-b06b5db78bab admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:01:49.882348 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-a6bb2d2f-04a7-41ab-b37a-b06b5db78bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:01:49.904830 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:49.943223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb2d2f-04a7-41ab-b37a-b06b5db78bab admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:01:49.947320 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95094e22-b85f-4e31-a433-21d4f7dc1468 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:01:49.947966 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95094e22-b85f-4e31-a433-21d4f7dc1468 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:01:49.948291 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95094e22-b85f-4e31-a433-21d4f7dc1468 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:01:49.948579 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95094e22-b85f-4e31-a433-21d4f7dc1468 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:01:49.957682 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95094e22-b85f-4e31-a433-21d4f7dc1468 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:01:49.959149 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95094e22-b85f-4e31-a433-21d4f7dc1468 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:01:49.968277 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1392/2763] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:01:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:01:59.867994 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:01:59.964229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=17 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:02:11.729730 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d634a0-2bfb-420c-9084-086e347b0744 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:02:11.796141 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d634a0-2bfb-420c-9084-086e347b0744 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:02:11.801575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:02:11.802454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:02:11.802732 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:02:11.803002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] PATH_INFO: `/v3/users/ddbbe75ee31e4725a8de89ab9c234125` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:02:11.804303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddbbe75ee31e4725a8de89ab9c234125)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:02:11.806275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:02:11.854767 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddbbe75ee31e4725a8de89ab9c234125'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:02:11.863150 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddbbe75ee31e4725a8de89ab9c234125'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:02:11.863749 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddbbe75ee31e4725a8de89ab9c234125'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:02:11.870587 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddbbe75ee31e4725a8de89ab9c234125'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:02:11.881993 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddbbe75ee31e4725a8de89ab9c234125'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:02:11.887814 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1372/2764] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:02:11 2022] DELETE /identity/v3/users/ddbbe75ee31e4725a8de89ab9c234125 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:02:11.892677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f82ba-4eaf-43c4-9106-01e281b6dea0 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:02:11.893187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f82ba-4eaf-43c4-9106-01e281b6dea0 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:02:11.893299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f82ba-4eaf-43c4-9106-01e281b6dea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:02:11.911630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-1d8f82ba-4eaf-43c4-9106-01e281b6dea0 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:02:11.913235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1d8f82ba-4eaf-43c4-9106-01e281b6dea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:02:11.976168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1393/2765] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:02:11.986384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:02:11.988166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:02:12.046905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d3652-e7b6-4ae5-834c-8d4dd10f0f04 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:02:12.050881 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b3122-b624-450c-98a7-bf48fbbd69d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:02:12.051513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b3122-b624-450c-98a7-bf48fbbd69d6 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:02:12.051792 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b3122-b624-450c-98a7-bf48fbbd69d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:02:12.052086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b3122-b624-450c-98a7-bf48fbbd69d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:02:12.061254 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b3122-b624-450c-98a7-bf48fbbd69d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:02:12.062685 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b3122-b624-450c-98a7-bf48fbbd69d6 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:02:12.072301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1373/2766] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:02:12.341474 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f82ba-4eaf-43c4-9106-01e281b6dea0 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:02:12.343049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-1d8f82ba-4eaf-43c4-9106-01e281b6dea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:02:12.357447 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f82ba-4eaf-43c4-9106-01e281b6dea0 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:02:12.363105 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:02:12.364052 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:02:12.364336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:02:12.364635 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] PATH_INFO: `/v3/projects/a2953090fcd9480680957f0f9a16a38d` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:02:12.365887 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2953090fcd9480680957f0f9a16a38d)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:02:12.368060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:02:12.423286 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2953090fcd9480680957f0f9a16a38d', 'request_id': 'req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:02:12.434242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2953090fcd9480680957f0f9a16a38d', 'request_id': 'req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:02:12.440697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2953090fcd9480680957f0f9a16a38d', 'request_id': 'req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:02:12.451192 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] The token cache is being invalidate because project a2953090fcd9480680957f0f9a16a38d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:02:12.451484 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:02:12.452156 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:02:12.453741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1394/2767] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:02:12 2022] DELETE /identity/v3/projects/a2953090fcd9480680957f0f9a16a38d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:02:22.068145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:02:22.448687 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:02:39.780599 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1201ed9-3bef-4855-9e42-b8074c7a7efc None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:02:39.781176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1201ed9-3bef-4855-9e42-b8074c7a7efc None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:02:39.781278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1201ed9-3bef-4855-9e42-b8074c7a7efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:02:39.798753 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b1201ed9-3bef-4855-9e42-b8074c7a7efc None None] MFA Rules not processed for user `0995483e7e7049bbb639ef3142192abe`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:02:39.799948 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b1201ed9-3bef-4855-9e42-b8074c7a7efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:02:39.855913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1374/2768] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:02:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:02:40.186985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:02:40.188741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:02:40.251276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee008a9-1cb9-4565-85ee-aebebc43a9f2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:02:40.255111 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab5a52-5d10-46e2-938f-ffc1e5c90b1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:02:40.255786 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab5a52-5d10-46e2-938f-ffc1e5c90b1d service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:02:40.256104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab5a52-5d10-46e2-938f-ffc1e5c90b1d service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:02:40.256384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab5a52-5d10-46e2-938f-ffc1e5c90b1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:02:40.258257 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0cab5a52-5d10-46e2-938f-ffc1e5c90b1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:02:40.268472 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab5a52-5d10-46e2-938f-ffc1e5c90b1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:02:40.269927 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cab5a52-5d10-46e2-938f-ffc1e5c90b1d service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:02:40.278280 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1395/2769] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:02:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:02:41.090258 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1201ed9-3bef-4855-9e42-b8074c7a7efc None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:02:41.092594 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b1201ed9-3bef-4855-9e42-b8074c7a7efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:02:41.172287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1201ed9-3bef-4855-9e42-b8074c7a7efc None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:02:41.177461 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6494c-ed23-409f-85a3-637b7f987269 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:02:41.178146 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6494c-ed23-409f-85a3-637b7f987269 service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:02:41.178431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6494c-ed23-409f-85a3-637b7f987269 service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:02:41.178707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6494c-ed23-409f-85a3-637b7f987269 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:02:41.180693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7ac6494c-ed23-409f-85a3-637b7f987269 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:02:41.192192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6494c-ed23-409f-85a3-637b7f987269 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:02:41.193776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6494c-ed23-409f-85a3-637b7f987269 service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:02:41.203662 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1375/2770] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:02:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:02:49.449067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab5a52-5d10-46e2-938f-ffc1e5c90b1d service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:02:49.450992 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0cab5a52-5d10-46e2-938f-ffc1e5c90b1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:02:49.533454 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab5a52-5d10-46e2-938f-ffc1e5c90b1d service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:02:49.539866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b84e5e-e333-44bd-8bca-960cc1448e19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:02:49.540544 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b84e5e-e333-44bd-8bca-960cc1448e19 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:02:49.540832 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b84e5e-e333-44bd-8bca-960cc1448e19 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:02:49.541108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b84e5e-e333-44bd-8bca-960cc1448e19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:02:49.544571 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-32b84e5e-e333-44bd-8bca-960cc1448e19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:02:49.555270 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b84e5e-e333-44bd-8bca-960cc1448e19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:02:49.556903 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b84e5e-e333-44bd-8bca-960cc1448e19 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:02:49.567811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1396/2771] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:02:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:02:51.199345 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=57 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:02:59.563274 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=60 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:03:26.089876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6494c-ed23-409f-85a3-637b7f987269 service placement] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:03:26.091303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7ac6494c-ed23-409f-85a3-637b7f987269 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:03:26.103682 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6494c-ed23-409f-85a3-637b7f987269 service placement] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:03:26.108275 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197195f-2dab-4aa2-946e-18f155e2a874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:03:26.108929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197195f-2dab-4aa2-946e-18f155e2a874 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:03:26.109213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197195f-2dab-4aa2-946e-18f155e2a874 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:03:26.109521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197195f-2dab-4aa2-946e-18f155e2a874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:03:26.111585 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-5197195f-2dab-4aa2-946e-18f155e2a874 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:03:26.122504 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197195f-2dab-4aa2-946e-18f155e2a874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:03:26.123958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197195f-2dab-4aa2-946e-18f155e2a874 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:03:26.132466 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1376/2772] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:03:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:03:26.225116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b84e5e-e333-44bd-8bca-960cc1448e19 service nova] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:03:26.234905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b84e5e-e333-44bd-8bca-960cc1448e19 service nova] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:03:26.238907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804c3973-d1fa-4550-9756-71bcbdc69973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:03:26.239561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c3973-d1fa-4550-9756-71bcbdc69973 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:03:26.239847 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c3973-d1fa-4550-9756-71bcbdc69973 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:03:26.240154 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804c3973-d1fa-4550-9756-71bcbdc69973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:03:26.242016 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-804c3973-d1fa-4550-9756-71bcbdc69973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:03:26.256360 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c3973-d1fa-4550-9756-71bcbdc69973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:03:26.258009 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804c3973-d1fa-4550-9756-71bcbdc69973 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:03:26.268942 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1397/2773] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:03:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:03:36.129309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=6 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:03:36.264454 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=6 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:04:10.359315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197195f-2dab-4aa2-946e-18f155e2a874 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:04:10.371931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197195f-2dab-4aa2-946e-18f155e2a874 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:04:10.377346 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74e99c-d7b8-4d5b-a7aa-3c3fc65b577d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:04:10.378000 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74e99c-d7b8-4d5b-a7aa-3c3fc65b577d service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:04:10.378287 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74e99c-d7b8-4d5b-a7aa-3c3fc65b577d service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:04:10.378564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74e99c-d7b8-4d5b-a7aa-3c3fc65b577d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:04:10.380541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ba74e99c-d7b8-4d5b-a7aa-3c3fc65b577d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:04:10.396208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74e99c-d7b8-4d5b-a7aa-3c3fc65b577d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:04:10.397655 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74e99c-d7b8-4d5b-a7aa-3c3fc65b577d service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:04:10.409742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1377/2774] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:04:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:04:20.403931 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=6 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:04:20.736273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abf285-59d9-4dd1-8e3e-c23521c5c5ec None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:04:20.736797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abf285-59d9-4dd1-8e3e-c23521c5c5ec None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:04:20.736797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abf285-59d9-4dd1-8e3e-c23521c5c5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:04:20.754911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-29abf285-59d9-4dd1-8e3e-c23521c5c5ec None None] MFA Rules not processed for user `0995483e7e7049bbb639ef3142192abe`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:04:20.756173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-29abf285-59d9-4dd1-8e3e-c23521c5c5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:04:20.763853 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1398/2775] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:04:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:04:30.749506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:04:33.140475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74e99c-d7b8-4d5b-a7aa-3c3fc65b577d service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:04:33.142106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-ba74e99c-d7b8-4d5b-a7aa-3c3fc65b577d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:04:33.155865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74e99c-d7b8-4d5b-a7aa-3c3fc65b577d service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:04:33.161543 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de495dbf-2460-4092-9193-a6755ae3803a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:04:33.162168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de495dbf-2460-4092-9193-a6755ae3803a service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:04:33.162571 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de495dbf-2460-4092-9193-a6755ae3803a service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:04:33.162850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de495dbf-2460-4092-9193-a6755ae3803a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:04:33.164651 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-de495dbf-2460-4092-9193-a6755ae3803a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:04:33.177476 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de495dbf-2460-4092-9193-a6755ae3803a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:04:33.179108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de495dbf-2460-4092-9193-a6755ae3803a service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:04:33.190040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1378/2776] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:04:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:04:43.185530 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=6 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:05:25.134346 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abf285-59d9-4dd1-8e3e-c23521c5c5ec None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:05:25.146305 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abf285-59d9-4dd1-8e3e-c23521c5c5ec None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:05:25.151285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cef22-0bbd-437d-b716-967bd89db53b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:05:25.151933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cef22-0bbd-437d-b716-967bd89db53b service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:05:25.152265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cef22-0bbd-437d-b716-967bd89db53b service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:05:25.152543 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cef22-0bbd-437d-b716-967bd89db53b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:05:25.154775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-fe7cef22-0bbd-437d-b716-967bd89db53b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:05:25.219402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7cef22-0bbd-437d-b716-967bd89db53b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:05:25.220801 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7cef22-0bbd-437d-b716-967bd89db53b service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:05:25.230120 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1399/2777] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:05:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:05:26.163340 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de495dbf-2460-4092-9193-a6755ae3803a service nova] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:05:26.174426 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de495dbf-2460-4092-9193-a6755ae3803a service nova] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:05:26.178482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b1e34-552d-416d-834d-ea669b60b5ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:05:26.180278 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b1e34-552d-416d-834d-ea669b60b5ac service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:05:26.181061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b1e34-552d-416d-834d-ea669b60b5ac service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:05:26.181343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b1e34-552d-416d-834d-ea669b60b5ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:05:26.183272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f81b1e34-552d-416d-834d-ea669b60b5ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:05:26.197030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b1e34-552d-416d-834d-ea669b60b5ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:05:26.199511 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b1e34-552d-416d-834d-ea669b60b5ac service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:05:26.209703 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1379/2778] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:05:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 04:05:27.432572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cef22-0bbd-437d-b716-967bd89db53b service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:05:27.442282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cef22-0bbd-437d-b716-967bd89db53b service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:05:27.447069 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ed6e5-18bd-4894-9695-46f85ca0374b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:05:27.447722 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ed6e5-18bd-4894-9695-46f85ca0374b service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:05:27.448046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ed6e5-18bd-4894-9695-46f85ca0374b service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:05:27.448318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ed6e5-18bd-4894-9695-46f85ca0374b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:05:27.450684 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-386ed6e5-18bd-4894-9695-46f85ca0374b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:05:27.464827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ed6e5-18bd-4894-9695-46f85ca0374b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:05:27.466279 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ed6e5-18bd-4894-9695-46f85ca0374b service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:05:27.477634 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1400/2779] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:05:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:05:31.125816 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b1e34-552d-416d-834d-ea669b60b5ac service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:05:31.135858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b1e34-552d-416d-834d-ea669b60b5ac service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:05:31.139520 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462b7a0-0660-421f-9d36-e5267d465d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:05:31.140215 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462b7a0-0660-421f-9d36-e5267d465d28 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:05:31.140506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462b7a0-0660-421f-9d36-e5267d465d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:05:31.140785 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6462b7a0-0660-421f-9d36-e5267d465d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:05:31.142771 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6462b7a0-0660-421f-9d36-e5267d465d28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:05:31.215105 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462b7a0-0660-421f-9d36-e5267d465d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:05:31.216635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6462b7a0-0660-421f-9d36-e5267d465d28 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:05:31.228600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1380/2780] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:05:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:05:34.341564 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ed6e5-18bd-4894-9695-46f85ca0374b service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:05:34.355013 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ed6e5-18bd-4894-9695-46f85ca0374b service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:05:34.360302 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d365d-a4ff-4fc4-a940-f94a9854925a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:05:34.361714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d365d-a4ff-4fc4-a940-f94a9854925a service placement] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:05:34.362112 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d365d-a4ff-4fc4-a940-f94a9854925a service placement] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:05:34.362449 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d365d-a4ff-4fc4-a940-f94a9854925a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:05:34.364440 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b05d365d-a4ff-4fc4-a940-f94a9854925a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:05:34.377356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d365d-a4ff-4fc4-a940-f94a9854925a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:05:34.379066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d365d-a4ff-4fc4-a940-f94a9854925a service placement] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:05:34.389605 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1401/2781] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:05:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:05:37.561970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462b7a0-0660-421f-9d36-e5267d465d28 service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:05:37.564266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6462b7a0-0660-421f-9d36-e5267d465d28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:05:37.578016 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6462b7a0-0660-421f-9d36-e5267d465d28 service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:05:37.582907 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:05:37.583792 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:05:37.584096 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:05:37.584368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] PATH_INFO: `/v3/users/5991e17e8dfb4a59a4dd1a4da24ff90c` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:05:37.585552 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5991e17e8dfb4a59a4dd1a4da24ff90c)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:05:37.587472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:05:37.649171 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5991e17e8dfb4a59a4dd1a4da24ff90c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:37.657683 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5991e17e8dfb4a59a4dd1a4da24ff90c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:37.658307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5991e17e8dfb4a59a4dd1a4da24ff90c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:37.665314 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5991e17e8dfb4a59a4dd1a4da24ff90c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:37.678156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5991e17e8dfb4a59a4dd1a4da24ff90c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:37.684617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1381/2782] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:05:37 2022] DELETE /identity/v3/users/5991e17e8dfb4a59a4dd1a4da24ff90c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:05:37.873816 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d365d-a4ff-4fc4-a940-f94a9854925a service placement] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:05:37.885482 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d365d-a4ff-4fc4-a940-f94a9854925a service placement] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:05:37.946450 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:05:37.947336 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:05:37.947619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:05:37.947931 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] PATH_INFO: `/v3/projects/aac1a71ff09f4ae38a5c3704fc43a950` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:05:37.949573 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac1a71ff09f4ae38a5c3704fc43a950)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:05:37.951907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:05:38.010001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac1a71ff09f4ae38a5c3704fc43a950', 'request_id': 'req-65633697-f5dd-423c-9505-9d09ab58f58d'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.018797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac1a71ff09f4ae38a5c3704fc43a950', 'request_id': 'req-65633697-f5dd-423c-9505-9d09ab58f58d'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.025196 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac1a71ff09f4ae38a5c3704fc43a950', 'request_id': 'req-65633697-f5dd-423c-9505-9d09ab58f58d'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.035542 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] The token cache is being invalidate because project aac1a71ff09f4ae38a5c3704fc43a950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:05:38.035854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.036613 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.038198 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1402/2783] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:05:37 2022] DELETE /identity/v3/projects/aac1a71ff09f4ae38a5c3704fc43a950 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 04:05:38.042624 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:05:38.044057 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:05:38.057315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75903e7e-2313-4b27-98d6-c402c10b05d2 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:05:38.114272 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:05:38.125850 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:05:38.126938 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:05:38.127425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:05:38.127917 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] PATH_INFO: `/v3/users/525f7f33ec554dffbdad5d372805d0b2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:05:38.129253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=525f7f33ec554dffbdad5d372805d0b2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:05:38.131298 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:05:38.177858 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '525f7f33ec554dffbdad5d372805d0b2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.187010 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '525f7f33ec554dffbdad5d372805d0b2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.187708 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '525f7f33ec554dffbdad5d372805d0b2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.194898 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '525f7f33ec554dffbdad5d372805d0b2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.206542 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '525f7f33ec554dffbdad5d372805d0b2'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.212964 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1382/2784] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:05:38 2022] DELETE /identity/v3/users/525f7f33ec554dffbdad5d372805d0b2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:05:38.395049 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:05:38.396793 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:05:38.405630 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:05:38.412666 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65633697-f5dd-423c-9505-9d09ab58f58d admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:05:38.473355 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:05:38.474232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:05:38.474514 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:05:38.474788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] PATH_INFO: `/v3/projects/9340450b91db44459cc62a2276957230` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:05:38.476033 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9340450b91db44459cc62a2276957230)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:05:38.477956 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:05:38.525385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9340450b91db44459cc62a2276957230', 'request_id': 'req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.533491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9340450b91db44459cc62a2276957230', 'request_id': 'req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.539577 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9340450b91db44459cc62a2276957230', 'request_id': 'req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.550315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] The token cache is being invalidate because project 9340450b91db44459cc62a2276957230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:05:38.550624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.551243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:05:38.552797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1403/2785] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:05:38 2022] DELETE /identity/v3/projects/9340450b91db44459cc62a2276957230 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:05:48.209392 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:05:48.214866 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:05:48.218391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:05:48.293800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ce402-2b2d-4e99-a328-bf9776bb2ec4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:05:48.365438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a759f-e1f1-4126-ad80-b6469faf9dfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:05:48.366017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a759f-e1f1-4126-ad80-b6469faf9dfb service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:05:48.366309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a759f-e1f1-4126-ad80-b6469faf9dfb service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:05:48.366564 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a759f-e1f1-4126-ad80-b6469faf9dfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:05:48.368745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6f0a759f-e1f1-4126-ad80-b6469faf9dfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:05:48.429982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a759f-e1f1-4126-ad80-b6469faf9dfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:05:48.431617 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a759f-e1f1-4126-ad80-b6469faf9dfb service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:05:48.441480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1383/2786] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:05:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:05:48.547404 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:05:58.437175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=84 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:06:01.996151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:06:01.998087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:06:02.017999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21ed66-1255-42d2-99d3-f57bf1f96bc3 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:06:02.025728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5dde0-816a-4a91-9075-28f64877ad84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:06:02.027961 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5dde0-816a-4a91-9075-28f64877ad84 service nova] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:06:02.028980 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5dde0-816a-4a91-9075-28f64877ad84 service nova] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:06:02.029385 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5dde0-816a-4a91-9075-28f64877ad84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:06:02.033734 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-06e5dde0-816a-4a91-9075-28f64877ad84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:06:02.048636 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5dde0-816a-4a91-9075-28f64877ad84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:06:02.050468 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5dde0-816a-4a91-9075-28f64877ad84 service nova] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:06:02.063338 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1404/2787] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:06:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:06:05.025425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c740e-ede7-442d-887f-09d530db8f35 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:06:05.025952 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c740e-ede7-442d-887f-09d530db8f35 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:06:05.026068 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c740e-ede7-442d-887f-09d530db8f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:06:05.044955 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-452c740e-ede7-442d-887f-09d530db8f35 None None] MFA Rules not processed for user `0995483e7e7049bbb639ef3142192abe`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:06:05.046192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-452c740e-ede7-442d-887f-09d530db8f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:06:05.053888 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1384/2788] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:06:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:06:05.691008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5dde0-816a-4a91-9075-28f64877ad84 service nova] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:06:05.692528 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-06e5dde0-816a-4a91-9075-28f64877ad84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:06:05.762757 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5dde0-816a-4a91-9075-28f64877ad84 service nova] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:06:05.768338 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47a9b8-56dd-4bd1-b67d-d0d14311f797 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:06:05.768989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47a9b8-56dd-4bd1-b67d-d0d14311f797 service placement] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:06:05.769276 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47a9b8-56dd-4bd1-b67d-d0d14311f797 service placement] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:06:05.769558 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47a9b8-56dd-4bd1-b67d-d0d14311f797 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:06:05.771503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7a47a9b8-56dd-4bd1-b67d-d0d14311f797 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:06:05.783371 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47a9b8-56dd-4bd1-b67d-d0d14311f797 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:06:05.784989 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47a9b8-56dd-4bd1-b67d-d0d14311f797 service placement] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:06:05.795518 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1405/2789] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:06:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:06:15.039125 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:06:15.791393 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=38 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:07:47.317701 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c740e-ede7-442d-887f-09d530db8f35 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:47.382418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c740e-ede7-442d-887f-09d530db8f35 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:47.388679 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307a230-ea6d-4b55-b3dc-792d8921161a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:47.389633 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307a230-ea6d-4b55-b3dc-792d8921161a service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:47.389924 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307a230-ea6d-4b55-b3dc-792d8921161a service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:47.390463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307a230-ea6d-4b55-b3dc-792d8921161a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:47.392518 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c307a230-ea6d-4b55-b3dc-792d8921161a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:47.427984 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307a230-ea6d-4b55-b3dc-792d8921161a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:47.429667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307a230-ea6d-4b55-b3dc-792d8921161a service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:47.439723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1385/2790] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:07:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:07:53.410591 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47a9b8-56dd-4bd1-b67d-d0d14311f797 service placement] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:53.412278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7a47a9b8-56dd-4bd1-b67d-d0d14311f797 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:53.497827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47a9b8-56dd-4bd1-b67d-d0d14311f797 service placement] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:53.501897 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce092841-9e0e-4a40-aad2-99da7101fb0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:53.502534 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce092841-9e0e-4a40-aad2-99da7101fb0d service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:53.502812 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce092841-9e0e-4a40-aad2-99da7101fb0d service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:53.503145 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce092841-9e0e-4a40-aad2-99da7101fb0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:53.504945 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ce092841-9e0e-4a40-aad2-99da7101fb0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:53.607815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce092841-9e0e-4a40-aad2-99da7101fb0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:53.609342 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce092841-9e0e-4a40-aad2-99da7101fb0d service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:53.629456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1406/2791] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:07:56.760541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307a230-ea6d-4b55-b3dc-792d8921161a service placement] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:56.763125 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c307a230-ea6d-4b55-b3dc-792d8921161a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:56.777063 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307a230-ea6d-4b55-b3dc-792d8921161a service placement] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:56.783156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd0457-f5ff-4e63-be3c-4b5a63067a58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:56.783902 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd0457-f5ff-4e63-be3c-4b5a63067a58 service placement] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:56.784270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd0457-f5ff-4e63-be3c-4b5a63067a58 service placement] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:56.784604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd0457-f5ff-4e63-be3c-4b5a63067a58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:56.786401 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-81fd0457-f5ff-4e63-be3c-4b5a63067a58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:56.801101 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd0457-f5ff-4e63-be3c-4b5a63067a58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:56.803211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd0457-f5ff-4e63-be3c-4b5a63067a58 service placement] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:56.817350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1386/2792] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:07:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:07:56.822410 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce092841-9e0e-4a40-aad2-99da7101fb0d service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:56.823849 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ce092841-9e0e-4a40-aad2-99da7101fb0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:56.838066 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce092841-9e0e-4a40-aad2-99da7101fb0d service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:56.842957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579bb71b-13bb-46a7-9ddf-de8cef46c8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:56.843850 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bb71b-13bb-46a7-9ddf-de8cef46c8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:56.844172 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bb71b-13bb-46a7-9ddf-de8cef46c8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:56.844452 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579bb71b-13bb-46a7-9ddf-de8cef46c8ae admin admin] PATH_INFO: `/v3/users/0995483e7e7049bbb639ef3142192abe` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:56.848773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579bb71b-13bb-46a7-9ddf-de8cef46c8ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=0995483e7e7049bbb639ef3142192abe)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:56.851018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579bb71b-13bb-46a7-9ddf-de8cef46c8ae admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:56.915960 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-579bb71b-13bb-46a7-9ddf-de8cef46c8ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0995483e7e7049bbb639ef3142192abe'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:56.917026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-579bb71b-13bb-46a7-9ddf-de8cef46c8ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0995483e7e7049bbb639ef3142192abe'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:56.919705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:07:56.926929 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-579bb71b-13bb-46a7-9ddf-de8cef46c8ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0995483e7e7049bbb639ef3142192abe'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:56.934556 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-579bb71b-13bb-46a7-9ddf-de8cef46c8ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0995483e7e7049bbb639ef3142192abe'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:56.949409 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-579bb71b-13bb-46a7-9ddf-de8cef46c8ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0995483e7e7049bbb639ef3142192abe'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:56.956985 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1407/2793] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:07:56 2022] DELETE /identity/v3/users/0995483e7e7049bbb639ef3142192abe => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:07:57.144115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd0457-f5ff-4e63-be3c-4b5a63067a58 service placement] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:57.159891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd0457-f5ff-4e63-be3c-4b5a63067a58 service placement] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:57.218279 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47c3e4-f864-490b-a853-36b940e1bc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:57.219167 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47c3e4-f864-490b-a853-36b940e1bc59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:57.219446 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47c3e4-f864-490b-a853-36b940e1bc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:57.219719 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47c3e4-f864-490b-a853-36b940e1bc59 admin admin] PATH_INFO: `/v3/projects/5c29114118814a298f6337842ad29a9b` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:57.221043 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47c3e4-f864-490b-a853-36b940e1bc59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c29114118814a298f6337842ad29a9b)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:57.223017 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47c3e4-f864-490b-a853-36b940e1bc59 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:57.276452 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8a47c3e4-f864-490b-a853-36b940e1bc59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c29114118814a298f6337842ad29a9b', 'request_id': 'req-8a47c3e4-f864-490b-a853-36b940e1bc59'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:57.285007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8a47c3e4-f864-490b-a853-36b940e1bc59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c29114118814a298f6337842ad29a9b', 'request_id': 'req-8a47c3e4-f864-490b-a853-36b940e1bc59'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:57.290754 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8a47c3e4-f864-490b-a853-36b940e1bc59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c29114118814a298f6337842ad29a9b', 'request_id': 'req-8a47c3e4-f864-490b-a853-36b940e1bc59'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:57.297974 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:07:57.303071 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8a47c3e4-f864-490b-a853-36b940e1bc59 admin admin] The token cache is being invalidate because project 5c29114118814a298f6337842ad29a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:07:57.303462 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8a47c3e4-f864-490b-a853-36b940e1bc59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:57.304414 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8a47c3e4-f864-490b-a853-36b940e1bc59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:57.307054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1387/2794] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:07:57 2022] DELETE /identity/v3/projects/5c29114118814a298f6337842ad29a9b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:07:57.312499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348cecb-7d89-4ef7-9ecd-781c58c7d79b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:57.312898 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348cecb-7d89-4ef7-9ecd-781c58c7d79b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:57.313182 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348cecb-7d89-4ef7-9ecd-781c58c7d79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:57.370140 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-c348cecb-7d89-4ef7-9ecd-781c58c7d79b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:07:57.371271 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c348cecb-7d89-4ef7-9ecd-781c58c7d79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:57.437498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1408/2795] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:07:57.445757 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33565254-8471-46dc-8950-1cf572d7baef None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:57.446035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33565254-8471-46dc-8950-1cf572d7baef None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:57.446315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33565254-8471-46dc-8950-1cf572d7baef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:57.466370 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-33565254-8471-46dc-8950-1cf572d7baef None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:07:57.467580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-33565254-8471-46dc-8950-1cf572d7baef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:57.476693 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1388/2796] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:07:57.481062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348cecb-7d89-4ef7-9ecd-781c58c7d79b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:57.491676 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348cecb-7d89-4ef7-9ecd-781c58c7d79b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:57.497426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d0473-e22c-4557-81d9-42185c70245f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:57.498098 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d0473-e22c-4557-81d9-42185c70245f admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:57.498431 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d0473-e22c-4557-81d9-42185c70245f admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:57.498640 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d0473-e22c-4557-81d9-42185c70245f admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:57.499370 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d0473-e22c-4557-81d9-42185c70245f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 04:07:57.499678 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d0473-e22c-4557-81d9-42185c70245f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:57.501394 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d0473-e22c-4557-81d9-42185c70245f admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:57.513851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1409/2797] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 04:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 04:07:57.519536 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33565254-8471-46dc-8950-1cf572d7baef None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:57.530231 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33565254-8471-46dc-8950-1cf572d7baef None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:57.536449 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31fcde-1001-4287-b3f2-cf953f95a370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:57.537144 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31fcde-1001-4287-b3f2-cf953f95a370 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:57.537405 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31fcde-1001-4287-b3f2-cf953f95a370 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:57.537713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31fcde-1001-4287-b3f2-cf953f95a370 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:57.538408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31fcde-1001-4287-b3f2-cf953f95a370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:57.540400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31fcde-1001-4287-b3f2-cf953f95a370 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:57.569575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1389/2798] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:07:57 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:07:57.573691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d0473-e22c-4557-81d9-42185c70245f admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:57.584715 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d0473-e22c-4557-81d9-42185c70245f admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:57.647957 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ef8fa-75d6-457e-bcab-4ff9acb930a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:57.648584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ef8fa-75d6-457e-bcab-4ff9acb930a9 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:57.648868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ef8fa-75d6-457e-bcab-4ff9acb930a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:57.649141 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ef8fa-75d6-457e-bcab-4ff9acb930a9 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:57.649825 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ef8fa-75d6-457e-bcab-4ff9acb930a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:57.651778 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ef8fa-75d6-457e-bcab-4ff9acb930a9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:57.673834 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1410/2799] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:07:57 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:07:57.678084 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31fcde-1001-4287-b3f2-cf953f95a370 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:57.688837 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31fcde-1001-4287-b3f2-cf953f95a370 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:57.693561 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684ed60-f3a2-4aaa-b095-c8d7c71ea0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:57.694228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684ed60-f3a2-4aaa-b095-c8d7c71ea0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:57.694512 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684ed60-f3a2-4aaa-b095-c8d7c71ea0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:57.694796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684ed60-f3a2-4aaa-b095-c8d7c71ea0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:57.695515 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684ed60-f3a2-4aaa-b095-c8d7c71ea0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:57.697166 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684ed60-f3a2-4aaa-b095-c8d7c71ea0e4 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:57.707822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1390/2800] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:07:57.711815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ef8fa-75d6-457e-bcab-4ff9acb930a9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:57.722551 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ef8fa-75d6-457e-bcab-4ff9acb930a9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:57.727717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746b87e-3c42-4864-9821-86265293cf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:57.728430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746b87e-3c42-4864-9821-86265293cf0d admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:57.728720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746b87e-3c42-4864-9821-86265293cf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:57.728998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746b87e-3c42-4864-9821-86265293cf0d admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:57.729739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746b87e-3c42-4864-9821-86265293cf0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:57.731391 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746b87e-3c42-4864-9821-86265293cf0d admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:57.736717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:07:57.742907 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1411/2801] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:07:57.747431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684ed60-f3a2-4aaa-b095-c8d7c71ea0e4 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:57.758191 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684ed60-f3a2-4aaa-b095-c8d7c71ea0e4 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:57.763639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebeb29a-a4ea-42eb-b450-3c1bbc9afad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:57.764684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebeb29a-a4ea-42eb-b450-3c1bbc9afad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:57.764968 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebeb29a-a4ea-42eb-b450-3c1bbc9afad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:57.765244 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebeb29a-a4ea-42eb-b450-3c1bbc9afad3 admin admin] PATH_INFO: `/v3/projects/2da9c016d50e4a38a5ad1f2734240b2a/users/92e9abf6d1f447bcac5450c0970515df/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:57.781112 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebeb29a-a4ea-42eb-b450-3c1bbc9afad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da9c016d50e4a38a5ad1f2734240b2a, user_id=92e9abf6d1f447bcac5450c0970515df, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:57.783891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebeb29a-a4ea-42eb-b450-3c1bbc9afad3 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:57.799956 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1391/2802] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:07:57 2022] PUT /identity/v3/projects/2da9c016d50e4a38a5ad1f2734240b2a/users/92e9abf6d1f447bcac5450c0970515df/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:07:57.811223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2f2bf-aaaf-4ea7-adea-bace616bcfa6 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:57.811531 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2f2bf-aaaf-4ea7-adea-bace616bcfa6 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:57.811809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2f2bf-aaaf-4ea7-adea-bace616bcfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:57.838223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-c0f2f2bf-aaaf-4ea7-adea-bace616bcfa6 None None] MFA Rules not processed for user `92e9abf6d1f447bcac5450c0970515df`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:07:57.839443 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-c0f2f2bf-aaaf-4ea7-adea-bace616bcfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:57.904572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1412/2803] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:07:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:07:57.916173 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebeb29a-a4ea-42eb-b450-3c1bbc9afad3 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:57.918024 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-4ebeb29a-a4ea-42eb-b450-3c1bbc9afad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:57.995360 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebeb29a-a4ea-42eb-b450-3c1bbc9afad3 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:58.001717 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654ecd3-4ab5-435d-8cd9-8e7680b24e1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7138318e53ff4b7885942d49b19eb0a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:58.002368 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654ecd3-4ab5-435d-8cd9-8e7680b24e1c service nova] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:58.002645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654ecd3-4ab5-435d-8cd9-8e7680b24e1c service nova] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:58.002925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654ecd3-4ab5-435d-8cd9-8e7680b24e1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:58.010909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654ecd3-4ab5-435d-8cd9-8e7680b24e1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:58.012538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654ecd3-4ab5-435d-8cd9-8e7680b24e1c service nova] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:58.021864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1392/2804] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:07:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:07:58.259022 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2f2bf-aaaf-4ea7-adea-bace616bcfa6 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:58.326596 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2f2bf-aaaf-4ea7-adea-bace616bcfa6 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:58.331953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:58.332883 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:58.333168 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:58.333445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] PATH_INFO: `/v3/users/92e9abf6d1f447bcac5450c0970515df` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:58.334757 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92e9abf6d1f447bcac5450c0970515df)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:58.336774 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:58.393612 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92e9abf6d1f447bcac5450c0970515df'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:58.394627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92e9abf6d1f447bcac5450c0970515df'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:58.405001 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92e9abf6d1f447bcac5450c0970515df'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:58.413121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92e9abf6d1f447bcac5450c0970515df'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:58.425238 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92e9abf6d1f447bcac5450c0970515df'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:58.431268 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1413/2805] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:07:58 2022] DELETE /identity/v3/users/92e9abf6d1f447bcac5450c0970515df => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:07:58.436185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcb554-8972-466d-8414-e9482bab5973 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:58.436502 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcb554-8972-466d-8414-e9482bab5973 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:58.436800 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcb554-8972-466d-8414-e9482bab5973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:58.455513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1abcb554-8972-466d-8414-e9482bab5973 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:07:58.456751 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1abcb554-8972-466d-8414-e9482bab5973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:58.520726 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1393/2806] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:07:58.535233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:58.536868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:58.548587 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:07:58.598126 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decace54-32b2-4ee8-bd0f-f66a9d2a10a0 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:58.602470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2361d-0e59-432b-afde-6058e1736978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:58.603119 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2361d-0e59-432b-afde-6058e1736978 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:58.603383 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2361d-0e59-432b-afde-6058e1736978 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:58.603671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2361d-0e59-432b-afde-6058e1736978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:58.618075 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2361d-0e59-432b-afde-6058e1736978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:58.621823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2361d-0e59-432b-afde-6058e1736978 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:58.636269 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1414/2807] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:07:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:07:58.929599 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcb554-8972-466d-8414-e9482bab5973 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:58.931211 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1abcb554-8972-466d-8414-e9482bab5973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:58.955595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcb554-8972-466d-8414-e9482bab5973 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:58.962480 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eafa6a-cb6e-4225-ae51-e6a13103702c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:58.963338 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eafa6a-cb6e-4225-ae51-e6a13103702c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:58.963676 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eafa6a-cb6e-4225-ae51-e6a13103702c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:58.963913 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eafa6a-cb6e-4225-ae51-e6a13103702c admin admin] PATH_INFO: `/v3/projects/2da9c016d50e4a38a5ad1f2734240b2a` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:58.965328 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eafa6a-cb6e-4225-ae51-e6a13103702c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da9c016d50e4a38a5ad1f2734240b2a)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:58.967173 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eafa6a-cb6e-4225-ae51-e6a13103702c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:58.984136 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:07:59.026949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-46eafa6a-cb6e-4225-ae51-e6a13103702c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da9c016d50e4a38a5ad1f2734240b2a', 'request_id': 'req-46eafa6a-cb6e-4225-ae51-e6a13103702c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:59.034736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-46eafa6a-cb6e-4225-ae51-e6a13103702c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da9c016d50e4a38a5ad1f2734240b2a', 'request_id': 'req-46eafa6a-cb6e-4225-ae51-e6a13103702c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:59.040317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-46eafa6a-cb6e-4225-ae51-e6a13103702c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da9c016d50e4a38a5ad1f2734240b2a', 'request_id': 'req-46eafa6a-cb6e-4225-ae51-e6a13103702c'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:59.053833 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-46eafa6a-cb6e-4225-ae51-e6a13103702c admin admin] The token cache is being invalidate because project 2da9c016d50e4a38a5ad1f2734240b2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:07:59.054165 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-46eafa6a-cb6e-4225-ae51-e6a13103702c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:59.055188 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-46eafa6a-cb6e-4225-ae51-e6a13103702c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:07:59.057699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1394/2808] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:07:58 2022] DELETE /identity/v3/projects/2da9c016d50e4a38a5ad1f2734240b2a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:07:59.064406 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0873a3-f07e-4fe4-8129-1f861e09f0b1 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.064928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0873a3-f07e-4fe4-8129-1f861e09f0b1 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.065031 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0873a3-f07e-4fe4-8129-1f861e09f0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.083601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-eb0873a3-f07e-4fe4-8129-1f861e09f0b1 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:07:59.084836 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-eb0873a3-f07e-4fe4-8129-1f861e09f0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:59.146687 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1415/2809] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:07:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:07:59.154749 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155d010-fa46-4c6a-ba4e-668a85bfed7d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.155040 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155d010-fa46-4c6a-ba4e-668a85bfed7d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.155318 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8155d010-fa46-4c6a-ba4e-668a85bfed7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.175280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-8155d010-fa46-4c6a-ba4e-668a85bfed7d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:07:59.176541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8155d010-fa46-4c6a-ba4e-668a85bfed7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:59.185799 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1395/2810] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:07:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 04:07:59.190246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0873a3-f07e-4fe4-8129-1f861e09f0b1 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:59.200675 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0873a3-f07e-4fe4-8129-1f861e09f0b1 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:59.206341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8e42e-c698-49a2-9d46-b4a833031b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:59.207004 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8e42e-c698-49a2-9d46-b4a833031b24 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.207286 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8e42e-c698-49a2-9d46-b4a833031b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.207561 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8e42e-c698-49a2-9d46-b4a833031b24 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.208334 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8e42e-c698-49a2-9d46-b4a833031b24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 04:07:59.208641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8e42e-c698-49a2-9d46-b4a833031b24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:59.210333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8e42e-c698-49a2-9d46-b4a833031b24 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:59.222926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1416/2811] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 04:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 04:07:59.227281 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155d010-fa46-4c6a-ba4e-668a85bfed7d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:59.236681 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8155d010-fa46-4c6a-ba4e-668a85bfed7d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:59.241653 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e096cb-9704-4890-9076-3b89439338a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:59.242308 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e096cb-9704-4890-9076-3b89439338a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.242598 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e096cb-9704-4890-9076-3b89439338a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.242872 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e096cb-9704-4890-9076-3b89439338a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.243599 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e096cb-9704-4890-9076-3b89439338a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:59.245675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e096cb-9704-4890-9076-3b89439338a7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:59.273246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1396/2812] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:07:59 2022] POST /identity/v3/projects => generated 401 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:07:59.277625 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8e42e-c698-49a2-9d46-b4a833031b24 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:59.288369 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8e42e-c698-49a2-9d46-b4a833031b24 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:59.349917 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af193105-741e-4612-9d18-2c47cc5e1196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:59.350585 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af193105-741e-4612-9d18-2c47cc5e1196 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.350868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af193105-741e-4612-9d18-2c47cc5e1196 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.351151 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af193105-741e-4612-9d18-2c47cc5e1196 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.351826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af193105-741e-4612-9d18-2c47cc5e1196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:59.353770 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af193105-741e-4612-9d18-2c47cc5e1196 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:59.375341 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1417/2813] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:07:59 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:07:59.379775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e096cb-9704-4890-9076-3b89439338a7 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:59.390703 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e096cb-9704-4890-9076-3b89439338a7 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:59.396755 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb210f91-c659-4dc1-a251-e8a46b8b036b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:59.397450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb210f91-c659-4dc1-a251-e8a46b8b036b admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.397746 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb210f91-c659-4dc1-a251-e8a46b8b036b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.398033 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb210f91-c659-4dc1-a251-e8a46b8b036b admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.398750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb210f91-c659-4dc1-a251-e8a46b8b036b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:59.400388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb210f91-c659-4dc1-a251-e8a46b8b036b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:59.412585 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1397/2814] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:07:59.418465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af193105-741e-4612-9d18-2c47cc5e1196 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:59.434207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af193105-741e-4612-9d18-2c47cc5e1196 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:59.441121 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f831e1-1626-4b4c-a2a7-33429846afd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:59.441788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f831e1-1626-4b4c-a2a7-33429846afd8 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.442076 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f831e1-1626-4b4c-a2a7-33429846afd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.442354 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f831e1-1626-4b4c-a2a7-33429846afd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.443087 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f831e1-1626-4b4c-a2a7-33429846afd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:59.444714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f831e1-1626-4b4c-a2a7-33429846afd8 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:59.454864 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1418/2815] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:07:59.459090 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb210f91-c659-4dc1-a251-e8a46b8b036b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:59.469824 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb210f91-c659-4dc1-a251-e8a46b8b036b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:59.475455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cc595-ed5d-447e-854d-2f127b0210e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:59.476391 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cc595-ed5d-447e-854d-2f127b0210e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.476667 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cc595-ed5d-447e-854d-2f127b0210e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.476943 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cc595-ed5d-447e-854d-2f127b0210e8 admin admin] PATH_INFO: `/v3/projects/348f31bba3724048be0ace8b582c3fa1/users/fd0e7e9012e1494cbc5e64b62e42f4d8/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.494277 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cc595-ed5d-447e-854d-2f127b0210e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=348f31bba3724048be0ace8b582c3fa1, user_id=fd0e7e9012e1494cbc5e64b62e42f4d8, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:59.496978 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cc595-ed5d-447e-854d-2f127b0210e8 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:59.513692 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1398/2816] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:07:59 2022] PUT /identity/v3/projects/348f31bba3724048be0ace8b582c3fa1/users/fd0e7e9012e1494cbc5e64b62e42f4d8/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 04:07:59.524889 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d150c01-c4be-4ace-9eb6-5afed7b98cd7 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.525183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d150c01-c4be-4ace-9eb6-5afed7b98cd7 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.525463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d150c01-c4be-4ace-9eb6-5afed7b98cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.552699 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-8d150c01-c4be-4ace-9eb6-5afed7b98cd7 None None] MFA Rules not processed for user `fd0e7e9012e1494cbc5e64b62e42f4d8`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:07:59.553920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8d150c01-c4be-4ace-9eb6-5afed7b98cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:07:59.559265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:07:59.622621 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1419/2817] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:07:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 04:07:59.627438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cc595-ed5d-447e-854d-2f127b0210e8 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:59.692363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cc595-ed5d-447e-854d-2f127b0210e8 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:59.697743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7dcf7-7d63-4236-9422-a56fe62f78e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:59.698381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7dcf7-7d63-4236-9422-a56fe62f78e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.698665 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7dcf7-7d63-4236-9422-a56fe62f78e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.698940 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7dcf7-7d63-4236-9422-a56fe62f78e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.699625 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7dcf7-7d63-4236-9422-a56fe62f78e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:59.701659 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7dcf7-7d63-4236-9422-a56fe62f78e0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:59.733637 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1399/2818] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:07:59 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:07:59.737873 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d150c01-c4be-4ace-9eb6-5afed7b98cd7 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:59.748714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d150c01-c4be-4ace-9eb6-5afed7b98cd7 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:59.805866 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086e77e-47a1-401d-a372-ffa92c33fb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:59.806532 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086e77e-47a1-401d-a372-ffa92c33fb0e admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.806816 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086e77e-47a1-401d-a372-ffa92c33fb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.807097 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086e77e-47a1-401d-a372-ffa92c33fb0e admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.807762 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086e77e-47a1-401d-a372-ffa92c33fb0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:59.809740 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086e77e-47a1-401d-a372-ffa92c33fb0e admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:59.831213 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1420/2819] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:07:59 2022] POST /identity/v3/users => generated 306 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:07:59.835404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7dcf7-7d63-4236-9422-a56fe62f78e0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:59.845723 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7dcf7-7d63-4236-9422-a56fe62f78e0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:59.850649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea6e80-1294-4741-896c-19efbe95a268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:59.851291 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea6e80-1294-4741-896c-19efbe95a268 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.851575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea6e80-1294-4741-896c-19efbe95a268 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.851844 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea6e80-1294-4741-896c-19efbe95a268 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.852595 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea6e80-1294-4741-896c-19efbe95a268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:59.854193 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea6e80-1294-4741-896c-19efbe95a268 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:59.865782 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1400/2820] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:07:59.870163 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086e77e-47a1-401d-a372-ffa92c33fb0e admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:59.880715 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086e77e-47a1-401d-a372-ffa92c33fb0e admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:59.886095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d02af-47f6-4af1-b658-8f75672f3935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:59.887026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d02af-47f6-4af1-b658-8f75672f3935 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.887337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d02af-47f6-4af1-b658-8f75672f3935 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.887611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d02af-47f6-4af1-b658-8f75672f3935 admin admin] PATH_INFO: `/v3/projects/e48fc2f87f7447f68e3263f8b1eae6cc/users/f7c021610d444b1b9ee98c2af32f3d47/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.904026 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d02af-47f6-4af1-b658-8f75672f3935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e48fc2f87f7447f68e3263f8b1eae6cc, user_id=f7c021610d444b1b9ee98c2af32f3d47, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:07:59.906711 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d02af-47f6-4af1-b658-8f75672f3935 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:07:59.921900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1421/2821] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:07:59 2022] PUT /identity/v3/projects/e48fc2f87f7447f68e3263f8b1eae6cc/users/f7c021610d444b1b9ee98c2af32f3d47/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:07:59.925973 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea6e80-1294-4741-896c-19efbe95a268 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:07:59.936061 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea6e80-1294-4741-896c-19efbe95a268 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:07:59.945513 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:07:59.996707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d615b53-a597-44f1-9718-fd8c99fecc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:07:59.997316 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d615b53-a597-44f1-9718-fd8c99fecc36 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:07:59.997601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d615b53-a597-44f1-9718-fd8c99fecc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:07:59.997975 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d615b53-a597-44f1-9718-fd8c99fecc36 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:07:59.998709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d615b53-a597-44f1-9718-fd8c99fecc36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:00.000362 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d615b53-a597-44f1-9718-fd8c99fecc36 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:00.010386 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1401/2822] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:08:00.014797 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d02af-47f6-4af1-b658-8f75672f3935 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:00.025949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d02af-47f6-4af1-b658-8f75672f3935 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:00.031309 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afa90c-98ae-4d8a-afc8-bdb9db5c9c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:00.032353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afa90c-98ae-4d8a-afc8-bdb9db5c9c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:00.032635 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afa90c-98ae-4d8a-afc8-bdb9db5c9c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:00.032911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afa90c-98ae-4d8a-afc8-bdb9db5c9c00 admin admin] PATH_INFO: `/v3/projects/e48fc2f87f7447f68e3263f8b1eae6cc/users/f7c021610d444b1b9ee98c2af32f3d47/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:00.034998 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afa90c-98ae-4d8a-afc8-bdb9db5c9c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e48fc2f87f7447f68e3263f8b1eae6cc, user_id=f7c021610d444b1b9ee98c2af32f3d47, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:00.037706 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afa90c-98ae-4d8a-afc8-bdb9db5c9c00 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:00.048491 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1422/2823] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:08:00 2022] PUT /identity/v3/projects/e48fc2f87f7447f68e3263f8b1eae6cc/users/f7c021610d444b1b9ee98c2af32f3d47/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:00.060127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186e30d-6989-4820-b720-edee0a2a7b34 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:00.060424 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186e30d-6989-4820-b720-edee0a2a7b34 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:00.060700 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186e30d-6989-4820-b720-edee0a2a7b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:00.089091 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-c186e30d-6989-4820-b720-edee0a2a7b34 None None] MFA Rules not processed for user `f7c021610d444b1b9ee98c2af32f3d47`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:00.090297 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c186e30d-6989-4820-b720-edee0a2a7b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:00.164248 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1402/2824] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:00 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:00.175363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afa90c-98ae-4d8a-afc8-bdb9db5c9c00 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:00.177413 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d6afa90c-98ae-4d8a-afc8-bdb9db5c9c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:00.239003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afa90c-98ae-4d8a-afc8-bdb9db5c9c00 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:00.243311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609b6ba-eb2f-4516-af2e-3943c888ae68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:00.243953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609b6ba-eb2f-4516-af2e-3943c888ae68 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:00.244273 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609b6ba-eb2f-4516-af2e-3943c888ae68 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:00.244646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609b6ba-eb2f-4516-af2e-3943c888ae68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:00.255207 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609b6ba-eb2f-4516-af2e-3943c888ae68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:00.256653 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609b6ba-eb2f-4516-af2e-3943c888ae68 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:00.266611 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1423/2825] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:08:00 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:08:00.341200 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186e30d-6989-4820-b720-edee0a2a7b34 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:00.351829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186e30d-6989-4820-b720-edee0a2a7b34 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:00.355875 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d9264-cbec-452d-8496-57e0f389f7a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:00.356575 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d9264-cbec-452d-8496-57e0f389f7a7 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:00.356835 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d9264-cbec-452d-8496-57e0f389f7a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:00.357106 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d9264-cbec-452d-8496-57e0f389f7a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:00.420286 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d9264-cbec-452d-8496-57e0f389f7a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:00.421745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d9264-cbec-452d-8496-57e0f389f7a7 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:00.431546 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1403/2826] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:08:00 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 04:08:04.560681 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609b6ba-eb2f-4516-af2e-3943c888ae68 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:04.565483 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:04.625926 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609b6ba-eb2f-4516-af2e-3943c888ae68 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:04.630856 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:04.631741 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:04.632053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:04.632335 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] PATH_INFO: `/v3/users/fd0e7e9012e1494cbc5e64b62e42f4d8` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:04.634090 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd0e7e9012e1494cbc5e64b62e42f4d8)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:04.636890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:04.695173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd0e7e9012e1494cbc5e64b62e42f4d8'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:04.695855 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd0e7e9012e1494cbc5e64b62e42f4d8'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:04.704738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd0e7e9012e1494cbc5e64b62e42f4d8'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:04.712343 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd0e7e9012e1494cbc5e64b62e42f4d8'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:04.727720 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd0e7e9012e1494cbc5e64b62e42f4d8'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:04.734230 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1424/2827] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:08:04 2022] DELETE /identity/v3/users/fd0e7e9012e1494cbc5e64b62e42f4d8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:04.738959 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96644f71-63ab-41b8-b79f-c42146042b9b None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:04.739482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96644f71-63ab-41b8-b79f-c42146042b9b None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:04.739482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96644f71-63ab-41b8-b79f-c42146042b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:04.757284 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-96644f71-63ab-41b8-b79f-c42146042b9b None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:04.758488 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-96644f71-63ab-41b8-b79f-c42146042b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:04.789135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:04.821455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1404/2828] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:04.831809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:04.833454 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:04.892584 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03810bca-0a98-4d85-8d49-0d9e53adc492 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:04.897095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042462a-bba1-407e-905c-04fe5c043b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:04.897769 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042462a-bba1-407e-905c-04fe5c043b1c service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:04.898074 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042462a-bba1-407e-905c-04fe5c043b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:04.898327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042462a-bba1-407e-905c-04fe5c043b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:04.907822 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042462a-bba1-407e-905c-04fe5c043b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:04.909311 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042462a-bba1-407e-905c-04fe5c043b1c service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:04.919008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1425/2829] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:08:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:08:05.202335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96644f71-63ab-41b8-b79f-c42146042b9b None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:05.203936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-96644f71-63ab-41b8-b79f-c42146042b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:05.216255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96644f71-63ab-41b8-b79f-c42146042b9b None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:05.221620 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:05.222588 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:05.222868 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:05.223132 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] PATH_INFO: `/v3/projects/348f31bba3724048be0ace8b582c3fa1` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:05.224404 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=348f31bba3724048be0ace8b582c3fa1)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:05.226325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:05.274120 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '348f31bba3724048be0ace8b582c3fa1', 'request_id': 'req-7fceea41-3d0d-4839-b507-8addcf21ab96'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.281743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '348f31bba3724048be0ace8b582c3fa1', 'request_id': 'req-7fceea41-3d0d-4839-b507-8addcf21ab96'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.287207 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '348f31bba3724048be0ace8b582c3fa1', 'request_id': 'req-7fceea41-3d0d-4839-b507-8addcf21ab96'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.298227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] The token cache is being invalidate because project 348f31bba3724048be0ace8b582c3fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:08:05.298517 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.299138 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.300741 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1405/2830] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:08:05 2022] DELETE /identity/v3/projects/348f31bba3724048be0ace8b582c3fa1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:05.304811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042462a-bba1-407e-905c-04fe5c043b1c service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:05.314488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042462a-bba1-407e-905c-04fe5c043b1c service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:05.334786 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:05.373086 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04807450-e472-465a-a731-51343f779052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:05.374008 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04807450-e472-465a-a731-51343f779052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:05.374295 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04807450-e472-465a-a731-51343f779052 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:05.374575 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04807450-e472-465a-a731-51343f779052 admin admin] PATH_INFO: `/v3/users/f7c021610d444b1b9ee98c2af32f3d47` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:05.375843 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04807450-e472-465a-a731-51343f779052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7c021610d444b1b9ee98c2af32f3d47)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:05.377842 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04807450-e472-465a-a731-51343f779052 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:05.427430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-04807450-e472-465a-a731-51343f779052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7c021610d444b1b9ee98c2af32f3d47'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.428123 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-04807450-e472-465a-a731-51343f779052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7c021610d444b1b9ee98c2af32f3d47'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.436404 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-04807450-e472-465a-a731-51343f779052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7c021610d444b1b9ee98c2af32f3d47'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.443247 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-04807450-e472-465a-a731-51343f779052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7c021610d444b1b9ee98c2af32f3d47'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.455043 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-04807450-e472-465a-a731-51343f779052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7c021610d444b1b9ee98c2af32f3d47'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.460932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1426/2831] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:08:05 2022] DELETE /identity/v3/users/f7c021610d444b1b9ee98c2af32f3d47 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 04:08:05.742146 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:05.743671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:05.757118 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fceea41-3d0d-4839-b507-8addcf21ab96 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:05.816960 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:05.817836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:05.818115 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:05.818390 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193 admin admin] PATH_INFO: `/v3/projects/e48fc2f87f7447f68e3263f8b1eae6cc` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:05.819754 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e48fc2f87f7447f68e3263f8b1eae6cc)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:05.821707 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:05.870573 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e48fc2f87f7447f68e3263f8b1eae6cc', 'request_id': 'req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.881536 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e48fc2f87f7447f68e3263f8b1eae6cc', 'request_id': 'req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.883711 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:05.888847 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e48fc2f87f7447f68e3263f8b1eae6cc', 'request_id': 'req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.902288 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193 admin admin] The token cache is being invalidate because project e48fc2f87f7447f68e3263f8b1eae6cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:08:05.902580 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.903776 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c7cd60f3-1d21-40b9-83a3-e9c66bc5e193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:05.905455 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1406/2832] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:08:05 2022] DELETE /identity/v3/projects/e48fc2f87f7447f68e3263f8b1eae6cc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:05.910572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682e3c-34dd-46f3-a11f-cf10e4f52b83 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:05.911053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682e3c-34dd-46f3-a11f-cf10e4f52b83 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:05.911191 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682e3c-34dd-46f3-a11f-cf10e4f52b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:05.931672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-0d682e3c-34dd-46f3-a11f-cf10e4f52b83 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:05.933003 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-0d682e3c-34dd-46f3-a11f-cf10e4f52b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:05.994237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1427/2833] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:06.002367 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73320f2-9ab1-401f-b5a6-0b5449cbaeab None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.002657 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73320f2-9ab1-401f-b5a6-0b5449cbaeab None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.002935 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73320f2-9ab1-401f-b5a6-0b5449cbaeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.022031 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-f73320f2-9ab1-401f-b5a6-0b5449cbaeab None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:06.023236 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-f73320f2-9ab1-401f-b5a6-0b5449cbaeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:06.033038 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1407/2834] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:06.037288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682e3c-34dd-46f3-a11f-cf10e4f52b83 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:06.047386 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682e3c-34dd-46f3-a11f-cf10e4f52b83 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:06.053029 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc473bb7-519c-46e8-bf87-aa1369d304a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:06.053701 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc473bb7-519c-46e8-bf87-aa1369d304a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.053979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc473bb7-519c-46e8-bf87-aa1369d304a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.054257 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc473bb7-519c-46e8-bf87-aa1369d304a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.054990 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc473bb7-519c-46e8-bf87-aa1369d304a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 04:08:06.055326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc473bb7-519c-46e8-bf87-aa1369d304a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:06.056999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc473bb7-519c-46e8-bf87-aa1369d304a8 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:06.068892 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1428/2835] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 04:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 04:08:06.073124 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73320f2-9ab1-401f-b5a6-0b5449cbaeab None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:06.082770 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73320f2-9ab1-401f-b5a6-0b5449cbaeab None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:06.088764 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8aa95b-57f3-4566-8654-dc10a478fec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:06.089422 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aa95b-57f3-4566-8654-dc10a478fec0 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.089699 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aa95b-57f3-4566-8654-dc10a478fec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.089969 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aa95b-57f3-4566-8654-dc10a478fec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.090654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8aa95b-57f3-4566-8654-dc10a478fec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:06.092639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8aa95b-57f3-4566-8654-dc10a478fec0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:06.125128 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1408/2836] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:08:06 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:08:06.129223 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc473bb7-519c-46e8-bf87-aa1369d304a8 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:06.139855 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc473bb7-519c-46e8-bf87-aa1369d304a8 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:06.170179 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:06.199975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc1211-12b7-4319-93cc-d159a08fd5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:06.200700 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc1211-12b7-4319-93cc-d159a08fd5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.201017 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc1211-12b7-4319-93cc-d159a08fd5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.201301 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc1211-12b7-4319-93cc-d159a08fd5d6 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.202010 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc1211-12b7-4319-93cc-d159a08fd5d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:06.203973 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc1211-12b7-4319-93cc-d159a08fd5d6 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:06.227494 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1429/2837] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:08:06 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 04:08:06.231771 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8aa95b-57f3-4566-8654-dc10a478fec0 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:06.242725 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8aa95b-57f3-4566-8654-dc10a478fec0 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:06.247419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91baf36-18dc-4c85-90a5-8c74bf1c1c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:06.248054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91baf36-18dc-4c85-90a5-8c74bf1c1c5f admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.248341 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91baf36-18dc-4c85-90a5-8c74bf1c1c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.248614 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91baf36-18dc-4c85-90a5-8c74bf1c1c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.249348 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91baf36-18dc-4c85-90a5-8c74bf1c1c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:06.251088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91baf36-18dc-4c85-90a5-8c74bf1c1c5f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:06.262821 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1409/2838] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:08:06.266891 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc1211-12b7-4319-93cc-d159a08fd5d6 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:06.277789 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc1211-12b7-4319-93cc-d159a08fd5d6 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:06.283062 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1332989-0472-49b5-a5dc-5beb6ad9985c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:06.283733 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1332989-0472-49b5-a5dc-5beb6ad9985c admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.284041 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1332989-0472-49b5-a5dc-5beb6ad9985c admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.284333 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1332989-0472-49b5-a5dc-5beb6ad9985c admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.285104 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1332989-0472-49b5-a5dc-5beb6ad9985c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:06.286708 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1332989-0472-49b5-a5dc-5beb6ad9985c admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:06.297624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1430/2839] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:08:06.301926 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91baf36-18dc-4c85-90a5-8c74bf1c1c5f admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:06.312039 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91baf36-18dc-4c85-90a5-8c74bf1c1c5f admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:06.317238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2115b-cc9f-4258-bbfa-5a6600390855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:06.318113 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2115b-cc9f-4258-bbfa-5a6600390855 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.318388 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2115b-cc9f-4258-bbfa-5a6600390855 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.318658 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2115b-cc9f-4258-bbfa-5a6600390855 admin admin] PATH_INFO: `/v3/projects/f404c8975509452db7c935b90686cb6d/users/4520198104ff4de8aaac6f75b897d7ce/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.333574 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2115b-cc9f-4258-bbfa-5a6600390855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f404c8975509452db7c935b90686cb6d, user_id=4520198104ff4de8aaac6f75b897d7ce, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:06.336238 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2115b-cc9f-4258-bbfa-5a6600390855 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:06.355294 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1410/2840] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:08:06 2022] PUT /identity/v3/projects/f404c8975509452db7c935b90686cb6d/users/4520198104ff4de8aaac6f75b897d7ce/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:06.367264 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c96da2-0dbf-41c2-b876-d70c424e741c None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.367557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c96da2-0dbf-41c2-b876-d70c424e741c None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.367837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c96da2-0dbf-41c2-b876-d70c424e741c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.394871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-42c96da2-0dbf-41c2-b876-d70c424e741c None None] MFA Rules not processed for user `4520198104ff4de8aaac6f75b897d7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:06.396488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-42c96da2-0dbf-41c2-b876-d70c424e741c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:06.474736 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1431/2841] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:06.479694 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2115b-cc9f-4258-bbfa-5a6600390855 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:06.555697 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2115b-cc9f-4258-bbfa-5a6600390855 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:06.561065 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4367d-64a2-436b-ac35-1cfcbee22d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:06.561722 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4367d-64a2-436b-ac35-1cfcbee22d5c admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.562007 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4367d-64a2-436b-ac35-1cfcbee22d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.562282 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4367d-64a2-436b-ac35-1cfcbee22d5c admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.562925 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4367d-64a2-436b-ac35-1cfcbee22d5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:06.564939 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4367d-64a2-436b-ac35-1cfcbee22d5c admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:06.594109 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1411/2842] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:08:06 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:08:06.598315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c96da2-0dbf-41c2-b876-d70c424e741c None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:06.609608 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c96da2-0dbf-41c2-b876-d70c424e741c None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:06.668253 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8fdeb-bbe9-4779-b481-732239149b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:06.668932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8fdeb-bbe9-4779-b481-732239149b19 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.669219 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8fdeb-bbe9-4779-b481-732239149b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.669498 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8fdeb-bbe9-4779-b481-732239149b19 admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.670203 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8fdeb-bbe9-4779-b481-732239149b19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:06.672188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8fdeb-bbe9-4779-b481-732239149b19 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:06.693968 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1432/2843] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:08:06 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 04:08:06.698167 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4367d-64a2-436b-ac35-1cfcbee22d5c admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:06.708864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4367d-64a2-436b-ac35-1cfcbee22d5c admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:06.714463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dd838-3c5a-46e8-acbe-db426dd6b8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:06.715140 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dd838-3c5a-46e8-acbe-db426dd6b8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.715419 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dd838-3c5a-46e8-acbe-db426dd6b8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.715709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dd838-3c5a-46e8-acbe-db426dd6b8f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.716537 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0dd838-3c5a-46e8-acbe-db426dd6b8f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:06.718180 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0dd838-3c5a-46e8-acbe-db426dd6b8f1 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:06.729221 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1412/2844] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 04:08:06.733568 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8fdeb-bbe9-4779-b481-732239149b19 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:06.743801 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8fdeb-bbe9-4779-b481-732239149b19 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:06.748804 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4feab-22c5-4a6c-8c64-e5b477cf37bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:06.749693 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4feab-22c5-4a6c-8c64-e5b477cf37bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.749988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4feab-22c5-4a6c-8c64-e5b477cf37bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.750266 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4feab-22c5-4a6c-8c64-e5b477cf37bd admin admin] PATH_INFO: `/v3/projects/31abf5334b654f70ade41405e0fa3a9b/users/064c37a48ba546f4a4a6b8d78c1a0d3b/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.765487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4feab-22c5-4a6c-8c64-e5b477cf37bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=31abf5334b654f70ade41405e0fa3a9b, user_id=064c37a48ba546f4a4a6b8d78c1a0d3b, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:06.768199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee4feab-22c5-4a6c-8c64-e5b477cf37bd admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:06.782564 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1433/2845] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:08:06 2022] PUT /identity/v3/projects/31abf5334b654f70ade41405e0fa3a9b/users/064c37a48ba546f4a4a6b8d78c1a0d3b/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:06.786569 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dd838-3c5a-46e8-acbe-db426dd6b8f1 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:06.797027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dd838-3c5a-46e8-acbe-db426dd6b8f1 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:06.850536 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:06.857813 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51601c27-031f-48b7-af53-8aae5ba2f5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:06.858897 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51601c27-031f-48b7-af53-8aae5ba2f5bf admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.858897 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51601c27-031f-48b7-af53-8aae5ba2f5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.859199 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51601c27-031f-48b7-af53-8aae5ba2f5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.859938 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51601c27-031f-48b7-af53-8aae5ba2f5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:06.862195 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51601c27-031f-48b7-af53-8aae5ba2f5bf admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:06.872339 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1413/2846] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:08:06.876928 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4feab-22c5-4a6c-8c64-e5b477cf37bd admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:06.883810 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:06.887751 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4feab-22c5-4a6c-8c64-e5b477cf37bd admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:06.893899 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742d16a-3031-4df3-87ac-2b2cd11bec21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:06.894788 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742d16a-3031-4df3-87ac-2b2cd11bec21 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.895077 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742d16a-3031-4df3-87ac-2b2cd11bec21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.895709 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742d16a-3031-4df3-87ac-2b2cd11bec21 admin admin] PATH_INFO: `/v3/projects/31abf5334b654f70ade41405e0fa3a9b/users/064c37a48ba546f4a4a6b8d78c1a0d3b/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:06.897730 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742d16a-3031-4df3-87ac-2b2cd11bec21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31abf5334b654f70ade41405e0fa3a9b, user_id=064c37a48ba546f4a4a6b8d78c1a0d3b, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:06.900503 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742d16a-3031-4df3-87ac-2b2cd11bec21 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:06.910707 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1434/2847] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:08:06 2022] PUT /identity/v3/projects/31abf5334b654f70ade41405e0fa3a9b/users/064c37a48ba546f4a4a6b8d78c1a0d3b/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:06.975185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86aca6-c1b4-487d-ad9c-e768a8eefa16 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:06.975431 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86aca6-c1b4-487d-ad9c-e768a8eefa16 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:06.975743 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86aca6-c1b4-487d-ad9c-e768a8eefa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:07.004384 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-dc86aca6-c1b4-487d-ad9c-e768a8eefa16 None None] MFA Rules not processed for user `064c37a48ba546f4a4a6b8d78c1a0d3b`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:07.005612 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-dc86aca6-c1b4-487d-ad9c-e768a8eefa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:07.080303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1414/2848] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:07.092072 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742d16a-3031-4df3-87ac-2b2cd11bec21 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:07.094183 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-d742d16a-3031-4df3-87ac-2b2cd11bec21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:07.157326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742d16a-3031-4df3-87ac-2b2cd11bec21 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:07.164487 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab11c6-4b44-4072-8099-6bdfd8eb4e28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:07.165422 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab11c6-4b44-4072-8099-6bdfd8eb4e28 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:07.165813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab11c6-4b44-4072-8099-6bdfd8eb4e28 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:07.166160 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab11c6-4b44-4072-8099-6bdfd8eb4e28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:07.179242 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab11c6-4b44-4072-8099-6bdfd8eb4e28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:07.180739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab11c6-4b44-4072-8099-6bdfd8eb4e28 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:07.190727 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1435/2849] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:08:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:08:07.273998 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86aca6-c1b4-487d-ad9c-e768a8eefa16 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:07.283805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86aca6-c1b4-487d-ad9c-e768a8eefa16 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:07.287586 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9da72b-b680-474b-a581-c0992195b268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:07.288270 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9da72b-b680-474b-a581-c0992195b268 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:07.288556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9da72b-b680-474b-a581-c0992195b268 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:07.288828 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9da72b-b680-474b-a581-c0992195b268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:07.348239 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9da72b-b680-474b-a581-c0992195b268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:07.349671 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9da72b-b680-474b-a581-c0992195b268 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:07.359271 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1415/2850] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:08:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:08:12.141803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab11c6-4b44-4072-8099-6bdfd8eb4e28 service cinder] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:12.205107 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab11c6-4b44-4072-8099-6bdfd8eb4e28 service cinder] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:12.209837 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:12.210719 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:12.211004 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:12.211281 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] PATH_INFO: `/v3/users/4520198104ff4de8aaac6f75b897d7ce` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:12.212516 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4520198104ff4de8aaac6f75b897d7ce)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:12.214464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:12.263656 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4520198104ff4de8aaac6f75b897d7ce'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:12.264325 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4520198104ff4de8aaac6f75b897d7ce'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:12.273398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4520198104ff4de8aaac6f75b897d7ce'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:12.281176 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4520198104ff4de8aaac6f75b897d7ce'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:12.294272 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4520198104ff4de8aaac6f75b897d7ce'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:12.300715 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1436/2851] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:08:12 2022] DELETE /identity/v3/users/4520198104ff4de8aaac6f75b897d7ce => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:12.305418 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3907f6-4aa4-45d5-b2da-cb4437d320b5 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:12.305949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3907f6-4aa4-45d5-b2da-cb4437d320b5 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:12.305949 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3907f6-4aa4-45d5-b2da-cb4437d320b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:12.323544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6c3907f6-4aa4-45d5-b2da-cb4437d320b5 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:12.324768 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6c3907f6-4aa4-45d5-b2da-cb4437d320b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:12.387909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1416/2852] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:12.400315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:12.401946 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:12.417237 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:12.464235 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b41389-c09b-4171-9fb1-fc30d21fb2f3 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:12.468173 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3072f-c2d1-48bd-8ba9-9c1d76ff8490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:12.468819 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3072f-c2d1-48bd-8ba9-9c1d76ff8490 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:12.469096 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3072f-c2d1-48bd-8ba9-9c1d76ff8490 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:12.469368 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3072f-c2d1-48bd-8ba9-9c1d76ff8490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:12.478686 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3072f-c2d1-48bd-8ba9-9c1d76ff8490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:12.480164 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3072f-c2d1-48bd-8ba9-9c1d76ff8490 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:12.490795 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1437/2853] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:08:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:08:12.797982 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3907f6-4aa4-45d5-b2da-cb4437d320b5 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:12.799591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6c3907f6-4aa4-45d5-b2da-cb4437d320b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:12.817759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3907f6-4aa4-45d5-b2da-cb4437d320b5 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:12.822902 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:12.823728 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:12.824027 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:12.824327 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] PATH_INFO: `/v3/projects/f404c8975509452db7c935b90686cb6d` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:12.825610 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f404c8975509452db7c935b90686cb6d)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:12.827540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:12.839647 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=100 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:12.877536 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f404c8975509452db7c935b90686cb6d', 'request_id': 'req-c2593d66-3e8b-4052-9f14-03b9959ffd30'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:12.886232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f404c8975509452db7c935b90686cb6d', 'request_id': 'req-c2593d66-3e8b-4052-9f14-03b9959ffd30'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:12.892174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f404c8975509452db7c935b90686cb6d', 'request_id': 'req-c2593d66-3e8b-4052-9f14-03b9959ffd30'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:12.903914 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] The token cache is being invalidate because project f404c8975509452db7c935b90686cb6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:08:12.904232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:12.904933 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:12.906593 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1417/2854] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:08:12 2022] DELETE /identity/v3/projects/f404c8975509452db7c935b90686cb6d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:12.911042 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3072f-c2d1-48bd-8ba9-9c1d76ff8490 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:12.921304 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3072f-c2d1-48bd-8ba9-9c1d76ff8490 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:12.989044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb49c63-8704-4f81-9531-59764f4f05c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:12.989871 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb49c63-8704-4f81-9531-59764f4f05c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:12.990200 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb49c63-8704-4f81-9531-59764f4f05c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:12.990473 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb49c63-8704-4f81-9531-59764f4f05c9 admin admin] PATH_INFO: `/v3/users/064c37a48ba546f4a4a6b8d78c1a0d3b` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:12.991692 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb49c63-8704-4f81-9531-59764f4f05c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=064c37a48ba546f4a4a6b8d78c1a0d3b)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:12.993673 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb49c63-8704-4f81-9531-59764f4f05c9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:13.044505 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8fb49c63-8704-4f81-9531-59764f4f05c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '064c37a48ba546f4a4a6b8d78c1a0d3b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:13.045157 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8fb49c63-8704-4f81-9531-59764f4f05c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '064c37a48ba546f4a4a6b8d78c1a0d3b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:13.053511 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8fb49c63-8704-4f81-9531-59764f4f05c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '064c37a48ba546f4a4a6b8d78c1a0d3b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:13.060767 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8fb49c63-8704-4f81-9531-59764f4f05c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '064c37a48ba546f4a4a6b8d78c1a0d3b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:13.072415 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8fb49c63-8704-4f81-9531-59764f4f05c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '064c37a48ba546f4a4a6b8d78c1a0d3b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:13.078316 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1438/2855] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:08:12 2022] DELETE /identity/v3/users/064c37a48ba546f4a4a6b8d78c1a0d3b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:13.385427 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:13.387037 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:13.401135 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2593d66-3e8b-4052-9f14-03b9959ffd30 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:13.458865 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3a54f-7863-4453-a2bc-635925b76a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:13.459750 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3a54f-7863-4453-a2bc-635925b76a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:13.460232 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3a54f-7863-4453-a2bc-635925b76a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:13.460519 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3a54f-7863-4453-a2bc-635925b76a15 admin admin] PATH_INFO: `/v3/projects/31abf5334b654f70ade41405e0fa3a9b` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:13.461742 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3a54f-7863-4453-a2bc-635925b76a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31abf5334b654f70ade41405e0fa3a9b)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:13.463673 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3a54f-7863-4453-a2bc-635925b76a15 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:13.512744 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c2e3a54f-7863-4453-a2bc-635925b76a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31abf5334b654f70ade41405e0fa3a9b', 'request_id': 'req-c2e3a54f-7863-4453-a2bc-635925b76a15'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:13.521927 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c2e3a54f-7863-4453-a2bc-635925b76a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31abf5334b654f70ade41405e0fa3a9b', 'request_id': 'req-c2e3a54f-7863-4453-a2bc-635925b76a15'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:13.527876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c2e3a54f-7863-4453-a2bc-635925b76a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31abf5334b654f70ade41405e0fa3a9b', 'request_id': 'req-c2e3a54f-7863-4453-a2bc-635925b76a15'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:13.542945 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c2e3a54f-7863-4453-a2bc-635925b76a15 admin admin] The token cache is being invalidate because project 31abf5334b654f70ade41405e0fa3a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:08:13.543233 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c2e3a54f-7863-4453-a2bc-635925b76a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:13.543915 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c2e3a54f-7863-4453-a2bc-635925b76a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:13.545532 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1418/2856] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:08:13 2022] DELETE /identity/v3/projects/31abf5334b654f70ade41405e0fa3a9b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:13.550765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a80837-0ea8-4b63-8a30-82dcba2f7bc0 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:13.551420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a80837-0ea8-4b63-8a30-82dcba2f7bc0 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:13.551735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a80837-0ea8-4b63-8a30-82dcba2f7bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:13.571749 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-37a80837-0ea8-4b63-8a30-82dcba2f7bc0 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:13.573027 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-37a80837-0ea8-4b63-8a30-82dcba2f7bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:13.605756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:13.637908 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1439/2857] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:13.646247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32c44c-c864-4c31-a33a-4d1b9e2c8707 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:13.646532 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32c44c-c864-4c31-a33a-4d1b9e2c8707 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:13.646805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32c44c-c864-4c31-a33a-4d1b9e2c8707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:13.665381 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-2e32c44c-c864-4c31-a33a-4d1b9e2c8707 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:13.666582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-2e32c44c-c864-4c31-a33a-4d1b9e2c8707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:13.675335 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1419/2858] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:13.679619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a80837-0ea8-4b63-8a30-82dcba2f7bc0 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:13.689718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a80837-0ea8-4b63-8a30-82dcba2f7bc0 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:13.694798 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dc864-7d92-4163-b660-bc24aa095e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:13.695456 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dc864-7d92-4163-b660-bc24aa095e05 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:13.695739 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dc864-7d92-4163-b660-bc24aa095e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:13.696032 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dc864-7d92-4163-b660-bc24aa095e05 admin admin] PATH_INFO: `/v3/domains` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:13.696780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dc864-7d92-4163-b660-bc24aa095e05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86935) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 04:08:13.697093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dc864-7d92-4163-b660-bc24aa095e05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:13.698717 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dc864-7d92-4163-b660-bc24aa095e05 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:13.710549 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1440/2859] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 04:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 04:08:13.715078 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32c44c-c864-4c31-a33a-4d1b9e2c8707 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:13.724839 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32c44c-c864-4c31-a33a-4d1b9e2c8707 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:13.730319 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c038c87-5942-4315-83e2-5564922eaa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:13.730971 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c038c87-5942-4315-83e2-5564922eaa3e admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:13.731253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c038c87-5942-4315-83e2-5564922eaa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:13.731521 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c038c87-5942-4315-83e2-5564922eaa3e admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:13.732247 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c038c87-5942-4315-83e2-5564922eaa3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:13.734170 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c038c87-5942-4315-83e2-5564922eaa3e admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:13.762450 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1420/2860] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:08:13 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:08:13.766999 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dc864-7d92-4163-b660-bc24aa095e05 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:13.777581 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dc864-7d92-4163-b660-bc24aa095e05 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:13.836464 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aca384-1a24-42b0-adaf-e87e76adfb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:13.837118 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aca384-1a24-42b0-adaf-e87e76adfb8b admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:13.837398 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aca384-1a24-42b0-adaf-e87e76adfb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:13.837668 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aca384-1a24-42b0-adaf-e87e76adfb8b admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:13.838366 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aca384-1a24-42b0-adaf-e87e76adfb8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:13.840458 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aca384-1a24-42b0-adaf-e87e76adfb8b admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:13.863297 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1441/2861] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:08:13 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:08:13.865229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c038c87-5942-4315-83e2-5564922eaa3e admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:13.870788 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:13.877894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c038c87-5942-4315-83e2-5564922eaa3e admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:13.883912 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f01779-5523-449c-bafb-26ef562ae226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:13.884609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f01779-5523-449c-bafb-26ef562ae226 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:13.884887 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f01779-5523-449c-bafb-26ef562ae226 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:13.885187 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f01779-5523-449c-bafb-26ef562ae226 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:13.886192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f01779-5523-449c-bafb-26ef562ae226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:13.887805 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f01779-5523-449c-bafb-26ef562ae226 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:13.898505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1421/2862] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:08:13.902479 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aca384-1a24-42b0-adaf-e87e76adfb8b admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:13.913257 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aca384-1a24-42b0-adaf-e87e76adfb8b admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:13.918542 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade74c8-a983-4046-a7c3-57274d527dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:13.919199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade74c8-a983-4046-a7c3-57274d527dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:13.919470 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade74c8-a983-4046-a7c3-57274d527dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:13.919742 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade74c8-a983-4046-a7c3-57274d527dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:13.920507 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade74c8-a983-4046-a7c3-57274d527dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:13.922110 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade74c8-a983-4046-a7c3-57274d527dc2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:13.932924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1442/2863] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:08:13.937145 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f01779-5523-449c-bafb-26ef562ae226 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:13.947963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f01779-5523-449c-bafb-26ef562ae226 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:13.955296 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012af5e-67b3-4a99-aba0-f96db6507365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:13.956343 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012af5e-67b3-4a99-aba0-f96db6507365 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:13.956625 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012af5e-67b3-4a99-aba0-f96db6507365 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:13.956894 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012af5e-67b3-4a99-aba0-f96db6507365 admin admin] PATH_INFO: `/v3/projects/4fa1a9d0e4cf4f83956ff798ef347c1a/users/055ae456baa944998a7dee492b35b6aa/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:13.973729 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012af5e-67b3-4a99-aba0-f96db6507365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa1a9d0e4cf4f83956ff798ef347c1a, user_id=055ae456baa944998a7dee492b35b6aa, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:13.976485 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012af5e-67b3-4a99-aba0-f96db6507365 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:13.991829 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1422/2864] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:08:13 2022] PUT /identity/v3/projects/4fa1a9d0e4cf4f83956ff798ef347c1a/users/055ae456baa944998a7dee492b35b6aa/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:14.003411 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bd9a6-49dc-4074-88fd-7152e595718b None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:14.003674 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bd9a6-49dc-4074-88fd-7152e595718b None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:14.003972 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bd9a6-49dc-4074-88fd-7152e595718b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:14.031765 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-111bd9a6-49dc-4074-88fd-7152e595718b None None] MFA Rules not processed for user `055ae456baa944998a7dee492b35b6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:14.032924 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-111bd9a6-49dc-4074-88fd-7152e595718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:14.099084 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1443/2865] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:14.103540 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012af5e-67b3-4a99-aba0-f96db6507365 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:14.171549 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012af5e-67b3-4a99-aba0-f96db6507365 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:14.176410 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f45e4f-689a-49b7-9170-cfb32a1b0045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:14.177081 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f45e4f-689a-49b7-9170-cfb32a1b0045 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:14.177366 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f45e4f-689a-49b7-9170-cfb32a1b0045 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:14.177642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f45e4f-689a-49b7-9170-cfb32a1b0045 admin admin] PATH_INFO: `/v3/projects` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:14.178331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f45e4f-689a-49b7-9170-cfb32a1b0045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:14.180315 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f45e4f-689a-49b7-9170-cfb32a1b0045 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:14.214532 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1423/2866] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:08:14 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:08:14.217624 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bd9a6-49dc-4074-88fd-7152e595718b None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:14.230193 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bd9a6-49dc-4074-88fd-7152e595718b None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:14.260657 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:14.289780 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055617ca-4f3b-417d-a0db-ae853b231f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:14.290384 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055617ca-4f3b-417d-a0db-ae853b231f9b admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:14.290667 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055617ca-4f3b-417d-a0db-ae853b231f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:14.290940 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055617ca-4f3b-417d-a0db-ae853b231f9b admin admin] PATH_INFO: `/v3/users` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:14.291626 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055617ca-4f3b-417d-a0db-ae853b231f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:14.293608 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055617ca-4f3b-417d-a0db-ae853b231f9b admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:14.314540 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1444/2867] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:08:14 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:08:14.318639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f45e4f-689a-49b7-9170-cfb32a1b0045 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:14.329317 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f45e4f-689a-49b7-9170-cfb32a1b0045 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:14.334517 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9692db-aa0a-46e4-9b71-7e9d927a807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:14.335184 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9692db-aa0a-46e4-9b71-7e9d927a807d admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:14.335463 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9692db-aa0a-46e4-9b71-7e9d927a807d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:14.335736 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9692db-aa0a-46e4-9b71-7e9d927a807d admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:14.336495 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9692db-aa0a-46e4-9b71-7e9d927a807d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:14.338101 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9692db-aa0a-46e4-9b71-7e9d927a807d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:14.348890 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1424/2868] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:08:14.353199 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055617ca-4f3b-417d-a0db-ae853b231f9b admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:14.364060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055617ca-4f3b-417d-a0db-ae853b231f9b admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:14.369572 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a688e11-9fcb-4bf3-8a43-5cc935809615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:14.370522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a688e11-9fcb-4bf3-8a43-5cc935809615 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:14.370818 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a688e11-9fcb-4bf3-8a43-5cc935809615 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:14.371080 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a688e11-9fcb-4bf3-8a43-5cc935809615 admin admin] PATH_INFO: `/v3/projects/b765b4db5cbd4e00845029c8a937186b/users/1d1589524e1e471fa0384d105f68bbeb/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:14.386619 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a688e11-9fcb-4bf3-8a43-5cc935809615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b765b4db5cbd4e00845029c8a937186b, user_id=1d1589524e1e471fa0384d105f68bbeb, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:14.389332 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a688e11-9fcb-4bf3-8a43-5cc935809615 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:14.407128 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1445/2869] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:08:14 2022] PUT /identity/v3/projects/b765b4db5cbd4e00845029c8a937186b/users/1d1589524e1e471fa0384d105f68bbeb/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:14.411380 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9692db-aa0a-46e4-9b71-7e9d927a807d admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:14.422305 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9692db-aa0a-46e4-9b71-7e9d927a807d admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:14.483635 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be3f4a-dc67-42fb-82b4-78c914b93be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:14.484357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be3f4a-dc67-42fb-82b4-78c914b93be0 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:14.484639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be3f4a-dc67-42fb-82b4-78c914b93be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:14.484916 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be3f4a-dc67-42fb-82b4-78c914b93be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:14.485639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be3f4a-dc67-42fb-82b4-78c914b93be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:14.487309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be3f4a-dc67-42fb-82b4-78c914b93be0 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:14.497528 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1425/2870] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:08:14.501979 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a688e11-9fcb-4bf3-8a43-5cc935809615 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:14.515326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a688e11-9fcb-4bf3-8a43-5cc935809615 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:14.520726 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b93dd9-3e8b-4fe2-8427-93903d8fb9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:14.521627 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b93dd9-3e8b-4fe2-8427-93903d8fb9fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:14.521938 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b93dd9-3e8b-4fe2-8427-93903d8fb9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:14.522246 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b93dd9-3e8b-4fe2-8427-93903d8fb9fe admin admin] PATH_INFO: `/v3/projects/b765b4db5cbd4e00845029c8a937186b/users/1d1589524e1e471fa0384d105f68bbeb/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:14.524485 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b93dd9-3e8b-4fe2-8427-93903d8fb9fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b765b4db5cbd4e00845029c8a937186b, user_id=1d1589524e1e471fa0384d105f68bbeb, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:14.527188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b93dd9-3e8b-4fe2-8427-93903d8fb9fe admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:14.537814 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1446/2871] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:08:14 2022] PUT /identity/v3/projects/b765b4db5cbd4e00845029c8a937186b/users/1d1589524e1e471fa0384d105f68bbeb/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:14.552088 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a5deb-dd5e-43df-99b2-522bb84ea0a8 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:14.552363 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a5deb-dd5e-43df-99b2-522bb84ea0a8 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:14.552649 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a5deb-dd5e-43df-99b2-522bb84ea0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:14.581864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6c3a5deb-dd5e-43df-99b2-522bb84ea0a8 None None] MFA Rules not processed for user `1d1589524e1e471fa0384d105f68bbeb`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:14.583064 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6c3a5deb-dd5e-43df-99b2-522bb84ea0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:14.612164 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:14.671525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1426/2872] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:14.682860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b93dd9-3e8b-4fe2-8427-93903d8fb9fe admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:14.684729 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-62b93dd9-3e8b-4fe2-8427-93903d8fb9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:14.743900 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b93dd9-3e8b-4fe2-8427-93903d8fb9fe admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:14.747887 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efc005-c66e-45b4-9b16-9f8143d48100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:14.748546 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efc005-c66e-45b4-9b16-9f8143d48100 service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:14.748823 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efc005-c66e-45b4-9b16-9f8143d48100 service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:14.749099 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efc005-c66e-45b4-9b16-9f8143d48100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:14.758882 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efc005-c66e-45b4-9b16-9f8143d48100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:14.760353 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efc005-c66e-45b4-9b16-9f8143d48100 service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:14.771463 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1447/2873] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:08:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:08:15.141309 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a5deb-dd5e-43df-99b2-522bb84ea0a8 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:15.204506 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a5deb-dd5e-43df-99b2-522bb84ea0a8 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:15.210230 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:15.211185 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:15.211473 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:15.211749 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] PATH_INFO: `/v3/users/055ae456baa944998a7dee492b35b6aa` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:15.213214 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=055ae456baa944998a7dee492b35b6aa)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:15.215175 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:15.264965 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '055ae456baa944998a7dee492b35b6aa'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:15.274566 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '055ae456baa944998a7dee492b35b6aa'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:15.275253 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '055ae456baa944998a7dee492b35b6aa'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:15.282383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '055ae456baa944998a7dee492b35b6aa'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:15.294307 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '055ae456baa944998a7dee492b35b6aa'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:15.300645 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1427/2874] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:08:15 2022] DELETE /identity/v3/users/055ae456baa944998a7dee492b35b6aa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:15.305395 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd2489-874e-47ba-b8b3-4cb2663b7544 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:15.305723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd2489-874e-47ba-b8b3-4cb2663b7544 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:15.305991 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd2489-874e-47ba-b8b3-4cb2663b7544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:15.325494 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-ecbd2489-874e-47ba-b8b3-4cb2663b7544 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:15.326641 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ecbd2489-874e-47ba-b8b3-4cb2663b7544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:15.386499 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1448/2875] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:15.396812 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:15.398325 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:15.451011 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:15.484321 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909635bc-df9f-4d1a-b254-174da0020507 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:15.488228 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7abf5-dec0-437d-8277-f3310e9b714e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:15.488876 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7abf5-dec0-437d-8277-f3310e9b714e service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:15.489155 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7abf5-dec0-437d-8277-f3310e9b714e service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:15.489425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7abf5-dec0-437d-8277-f3310e9b714e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:15.498538 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7abf5-dec0-437d-8277-f3310e9b714e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:15.499963 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7abf5-dec0-437d-8277-f3310e9b714e service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:15.509383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1428/2876] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:08:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:08:15.805743 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd2489-874e-47ba-b8b3-4cb2663b7544 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:15.807288 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-ecbd2489-874e-47ba-b8b3-4cb2663b7544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:15.819868 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd2489-874e-47ba-b8b3-4cb2663b7544 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:15.825314 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:15.826192 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:15.826465 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:15.826758 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] PATH_INFO: `/v3/projects/4fa1a9d0e4cf4f83956ff798ef347c1a` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:15.828036 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa1a9d0e4cf4f83956ff798ef347c1a)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:15.830093 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:15.854905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:15.882053 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa1a9d0e4cf4f83956ff798ef347c1a', 'request_id': 'req-8751d4c9-978d-49c1-9c8f-eab28b2e5474'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:15.889680 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa1a9d0e4cf4f83956ff798ef347c1a', 'request_id': 'req-8751d4c9-978d-49c1-9c8f-eab28b2e5474'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:15.896724 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa1a9d0e4cf4f83956ff798ef347c1a', 'request_id': 'req-8751d4c9-978d-49c1-9c8f-eab28b2e5474'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:15.906690 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] The token cache is being invalidate because project 4fa1a9d0e4cf4f83956ff798ef347c1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:08:15.906976 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:15.907590 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:15.909240 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1449/2877] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:08:15 2022] DELETE /identity/v3/projects/4fa1a9d0e4cf4f83956ff798ef347c1a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:15.913271 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7abf5-dec0-437d-8277-f3310e9b714e service neutron] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:15.922929 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7abf5-dec0-437d-8277-f3310e9b714e service neutron] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:15.983086 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77279f13-38c4-4f26-9cc7-1bf1f335e184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:15.983992 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77279f13-38c4-4f26-9cc7-1bf1f335e184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:15.984301 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77279f13-38c4-4f26-9cc7-1bf1f335e184 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:15.984572 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77279f13-38c4-4f26-9cc7-1bf1f335e184 admin admin] PATH_INFO: `/v3/users/1d1589524e1e471fa0384d105f68bbeb` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:15.985882 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77279f13-38c4-4f26-9cc7-1bf1f335e184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d1589524e1e471fa0384d105f68bbeb)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:15.987842 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77279f13-38c4-4f26-9cc7-1bf1f335e184 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:16.038640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-77279f13-38c4-4f26-9cc7-1bf1f335e184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d1589524e1e471fa0384d105f68bbeb'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:16.047709 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-77279f13-38c4-4f26-9cc7-1bf1f335e184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d1589524e1e471fa0384d105f68bbeb'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:16.048432 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-77279f13-38c4-4f26-9cc7-1bf1f335e184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d1589524e1e471fa0384d105f68bbeb'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:16.055752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-77279f13-38c4-4f26-9cc7-1bf1f335e184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d1589524e1e471fa0384d105f68bbeb'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:16.068201 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-77279f13-38c4-4f26-9cc7-1bf1f335e184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d1589524e1e471fa0384d105f68bbeb'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:16.074371 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1429/2878] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:08:15 2022] DELETE /identity/v3/users/1d1589524e1e471fa0384d105f68bbeb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:16.368065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:16.369522 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:16.385402 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751d4c9-978d-49c1-9c8f-eab28b2e5474 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:16.448490 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1457c-3e77-4e46-99ef-8dff96926c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:16.449664 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1457c-3e77-4e46-99ef-8dff96926c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:16.449950 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1457c-3e77-4e46-99ef-8dff96926c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:16.450306 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1457c-3e77-4e46-99ef-8dff96926c72 admin admin] PATH_INFO: `/v3/projects/b765b4db5cbd4e00845029c8a937186b` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:16.452419 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1457c-3e77-4e46-99ef-8dff96926c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b765b4db5cbd4e00845029c8a937186b)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:16.455074 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1457c-3e77-4e46-99ef-8dff96926c72 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:16.508772 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-00b1457c-3e77-4e46-99ef-8dff96926c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b765b4db5cbd4e00845029c8a937186b', 'request_id': 'req-00b1457c-3e77-4e46-99ef-8dff96926c72'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:16.517359 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-00b1457c-3e77-4e46-99ef-8dff96926c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b765b4db5cbd4e00845029c8a937186b', 'request_id': 'req-00b1457c-3e77-4e46-99ef-8dff96926c72'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:16.523901 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-00b1457c-3e77-4e46-99ef-8dff96926c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b765b4db5cbd4e00845029c8a937186b', 'request_id': 'req-00b1457c-3e77-4e46-99ef-8dff96926c72'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:16.534880 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-00b1457c-3e77-4e46-99ef-8dff96926c72 admin admin] The token cache is being invalidate because project b765b4db5cbd4e00845029c8a937186b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:08:16.535161 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-00b1457c-3e77-4e46-99ef-8dff96926c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:16.535811 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-00b1457c-3e77-4e46-99ef-8dff96926c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:16.537403 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1450/2879] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:08:16 2022] DELETE /identity/v3/projects/b765b4db5cbd4e00845029c8a937186b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:16.543510 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae5880-d4b2-4774-80ee-96dbeabb9ab9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:16.544052 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae5880-d4b2-4774-80ee-96dbeabb9ab9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:16.544169 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae5880-d4b2-4774-80ee-96dbeabb9ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:16.562246 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-d8ae5880-d4b2-4774-80ee-96dbeabb9ab9 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:16.563479 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-d8ae5880-d4b2-4774-80ee-96dbeabb9ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:16.609646 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:16.634475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1430/2880] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:16.642526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37514170-312d-4694-85b1-656fe54d9f5e None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:16.642813 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37514170-312d-4694-85b1-656fe54d9f5e None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:16.643089 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37514170-312d-4694-85b1-656fe54d9f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:16.661901 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-37514170-312d-4694-85b1-656fe54d9f5e None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:16.663078 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-37514170-312d-4694-85b1-656fe54d9f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:16.671065 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1451/2881] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:16.675425 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae5880-d4b2-4774-80ee-96dbeabb9ab9 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:16.685587 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae5880-d4b2-4774-80ee-96dbeabb9ab9 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:16.691482 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa330c-6f2d-4bf6-8314-ca8557164916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:16.692203 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa330c-6f2d-4bf6-8314-ca8557164916 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:16.692486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa330c-6f2d-4bf6-8314-ca8557164916 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:16.692758 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa330c-6f2d-4bf6-8314-ca8557164916 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:16.693556 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa330c-6f2d-4bf6-8314-ca8557164916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 04:08:16.693851 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa330c-6f2d-4bf6-8314-ca8557164916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:16.695505 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa330c-6f2d-4bf6-8314-ca8557164916 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:16.707609 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1431/2882] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 04:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 04:08:16.711738 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37514170-312d-4694-85b1-656fe54d9f5e None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:16.721407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37514170-312d-4694-85b1-656fe54d9f5e None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:16.727318 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640d81a-7e62-4c3c-9332-46c6a48dbce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:16.727962 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640d81a-7e62-4c3c-9332-46c6a48dbce9 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:16.728278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640d81a-7e62-4c3c-9332-46c6a48dbce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:16.728548 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640d81a-7e62-4c3c-9332-46c6a48dbce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:16.729243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640d81a-7e62-4c3c-9332-46c6a48dbce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:16.731144 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640d81a-7e62-4c3c-9332-46c6a48dbce9 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:16.759167 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1452/2883] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:08:16 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:08:16.763229 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa330c-6f2d-4bf6-8314-ca8557164916 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:16.773639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa330c-6f2d-4bf6-8314-ca8557164916 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:16.830907 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9ce62-9ebf-4a3a-9d6e-c43da29a3b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:16.831565 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9ce62-9ebf-4a3a-9d6e-c43da29a3b56 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:16.831846 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9ce62-9ebf-4a3a-9d6e-c43da29a3b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:16.832151 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9ce62-9ebf-4a3a-9d6e-c43da29a3b56 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:16.832826 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9ce62-9ebf-4a3a-9d6e-c43da29a3b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:16.834745 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9ce62-9ebf-4a3a-9d6e-c43da29a3b56 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:16.856376 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1432/2884] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:08:16 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:08:16.860643 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640d81a-7e62-4c3c-9332-46c6a48dbce9 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:16.871060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640d81a-7e62-4c3c-9332-46c6a48dbce9 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:16.875940 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401fc3f-4f3c-4967-807d-a2125dfc20c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:16.876672 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401fc3f-4f3c-4967-807d-a2125dfc20c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:16.877046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401fc3f-4f3c-4967-807d-a2125dfc20c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:16.877326 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401fc3f-4f3c-4967-807d-a2125dfc20c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:16.878058 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401fc3f-4f3c-4967-807d-a2125dfc20c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:16.879695 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401fc3f-4f3c-4967-807d-a2125dfc20c1 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:16.885232 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:16.892243 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1453/2885] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 04:08:16.896870 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9ce62-9ebf-4a3a-9d6e-c43da29a3b56 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:16.907255 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9ce62-9ebf-4a3a-9d6e-c43da29a3b56 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:16.912932 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ce80f-40ca-497f-8d8e-7c26886d823d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:16.913544 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ce80f-40ca-497f-8d8e-7c26886d823d admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:16.913832 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ce80f-40ca-497f-8d8e-7c26886d823d admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:16.914108 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ce80f-40ca-497f-8d8e-7c26886d823d admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:16.914860 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ce80f-40ca-497f-8d8e-7c26886d823d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:16.916524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ce80f-40ca-497f-8d8e-7c26886d823d admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:16.926891 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1433/2886] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:08:16.931131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401fc3f-4f3c-4967-807d-a2125dfc20c1 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:16.942233 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401fc3f-4f3c-4967-807d-a2125dfc20c1 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:16.947601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051e46b-d306-46bd-a8ef-0b0fde79a576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:16.948541 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051e46b-d306-46bd-a8ef-0b0fde79a576 admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:16.948826 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051e46b-d306-46bd-a8ef-0b0fde79a576 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:16.949100 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051e46b-d306-46bd-a8ef-0b0fde79a576 admin admin] PATH_INFO: `/v3/projects/4693050281f74586aac401170780283f/users/0ab5deab2b7b45e594e1f61ba0496d63/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:16.967920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051e46b-d306-46bd-a8ef-0b0fde79a576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4693050281f74586aac401170780283f, user_id=0ab5deab2b7b45e594e1f61ba0496d63, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:16.971911 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051e46b-d306-46bd-a8ef-0b0fde79a576 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:16.988315 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1454/2887] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:08:16 2022] PUT /identity/v3/projects/4693050281f74586aac401170780283f/users/0ab5deab2b7b45e594e1f61ba0496d63/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:17.002825 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc37236-69ae-42af-8248-abc55b926eb9 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:17.003097 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc37236-69ae-42af-8248-abc55b926eb9 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:17.003495 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc37236-69ae-42af-8248-abc55b926eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:17.037036 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-6fc37236-69ae-42af-8248-abc55b926eb9 None None] MFA Rules not processed for user `0ab5deab2b7b45e594e1f61ba0496d63`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:17.038271 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-6fc37236-69ae-42af-8248-abc55b926eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:17.105143 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1434/2888] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:17.116817 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051e46b-d306-46bd-a8ef-0b0fde79a576 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:17.118800 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f051e46b-d306-46bd-a8ef-0b0fde79a576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:17.214649 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051e46b-d306-46bd-a8ef-0b0fde79a576 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:17.218748 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84df465-8e2d-46c8-845a-39b4bc7f01ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:17.219407 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84df465-8e2d-46c8-845a-39b4bc7f01ae service cinder] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:17.219691 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84df465-8e2d-46c8-845a-39b4bc7f01ae service cinder] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:17.219967 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84df465-8e2d-46c8-845a-39b4bc7f01ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:17.228920 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84df465-8e2d-46c8-845a-39b4bc7f01ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:17.230378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84df465-8e2d-46c8-845a-39b4bc7f01ae service cinder] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:17.239492 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1455/2889] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:08:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:08:17.260284 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc37236-69ae-42af-8248-abc55b926eb9 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:17.308213 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:17.329421 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc37236-69ae-42af-8248-abc55b926eb9 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:17.334337 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:17.335261 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:17.335547 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:17.335822 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] PATH_INFO: `/v3/users/0ab5deab2b7b45e594e1f61ba0496d63` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:17.337137 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ab5deab2b7b45e594e1f61ba0496d63)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:17.339090 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:17.395098 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ab5deab2b7b45e594e1f61ba0496d63'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:17.405601 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ab5deab2b7b45e594e1f61ba0496d63'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:17.406357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ab5deab2b7b45e594e1f61ba0496d63'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:17.414436 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ab5deab2b7b45e594e1f61ba0496d63'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:17.427958 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ab5deab2b7b45e594e1f61ba0496d63'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:17.434127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1435/2890] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:08:17 2022] DELETE /identity/v3/users/0ab5deab2b7b45e594e1f61ba0496d63 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:17.439012 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a35b2-2a3f-4666-b350-12e127aece8a None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:17.439375 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a35b2-2a3f-4666-b350-12e127aece8a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:17.439671 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a35b2-2a3f-4666-b350-12e127aece8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:17.457988 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-e38a35b2-2a3f-4666-b350-12e127aece8a None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:17.459188 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e38a35b2-2a3f-4666-b350-12e127aece8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:17.525515 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1456/2891] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:17.535400 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:17.537085 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:17.600372 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801dfdd-58bc-484f-8672-281eaa643291 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:17.604675 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7a725-e2f3-456b-95c7-2d0d9d1ffbc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:17.605345 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7a725-e2f3-456b-95c7-2d0d9d1ffbc0 service neutron] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:17.605622 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7a725-e2f3-456b-95c7-2d0d9d1ffbc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:17.605907 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7a725-e2f3-456b-95c7-2d0d9d1ffbc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:17.616464 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7a725-e2f3-456b-95c7-2d0d9d1ffbc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:17.617907 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7a725-e2f3-456b-95c7-2d0d9d1ffbc0 service neutron] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:17.627874 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1436/2892] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:08:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:08:17.912184 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a35b2-2a3f-4666-b350-12e127aece8a None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:17.913714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-e38a35b2-2a3f-4666-b350-12e127aece8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:17.926846 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a35b2-2a3f-4666-b350-12e127aece8a None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:17.932827 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:17.933714 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:17.933993 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:17.934272 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba admin admin] PATH_INFO: `/v3/projects/4693050281f74586aac401170780283f` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:17.936070 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=4693050281f74586aac401170780283f)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:17.938028 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:17.991489 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4693050281f74586aac401170780283f', 'request_id': 'req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:17.995323 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:18.000697 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4693050281f74586aac401170780283f', 'request_id': 'req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:18.008327 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4693050281f74586aac401170780283f', 'request_id': 'req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:18.020131 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba admin admin] The token cache is being invalidate because project 4693050281f74586aac401170780283f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86935) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:08:18.020420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:18.021174 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-ba1d01dd-2b5a-43bb-bd69-1281e52672ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:18.022860 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1457/2893] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:08:17 2022] DELETE /identity/v3/projects/4693050281f74586aac401170780283f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:18.028127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c8974-c5cc-46be-8f01-3ed8e8606797 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.028403 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c8974-c5cc-46be-8f01-3ed8e8606797 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.028679 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c8974-c5cc-46be-8f01-3ed8e8606797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.047869 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-b41c8974-c5cc-46be-8f01-3ed8e8606797 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:18.049102 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b41c8974-c5cc-46be-8f01-3ed8e8606797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:18.109780 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1437/2894] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:18.117728 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adfa36-031c-4142-9d14-ff1009734549 None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.118011 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adfa36-031c-4142-9d14-ff1009734549 None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.118285 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adfa36-031c-4142-9d14-ff1009734549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.136854 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-b8adfa36-031c-4142-9d14-ff1009734549 None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:18.138060 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-b8adfa36-031c-4142-9d14-ff1009734549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:18.147002 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1458/2895] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:18.151174 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c8974-c5cc-46be-8f01-3ed8e8606797 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:18.156241 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:18.163487 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c8974-c5cc-46be-8f01-3ed8e8606797 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:18.169192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4e6a3-6641-4703-83d0-1aa7d08ae025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:18.169880 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4e6a3-6641-4703-83d0-1aa7d08ae025 admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.170156 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4e6a3-6641-4703-83d0-1aa7d08ae025 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.170454 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4e6a3-6641-4703-83d0-1aa7d08ae025 admin admin] PATH_INFO: `/v3/domains` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.171179 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4e6a3-6641-4703-83d0-1aa7d08ae025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86934) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 04:08:18.171483 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4e6a3-6641-4703-83d0-1aa7d08ae025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:18.173168 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4e6a3-6641-4703-83d0-1aa7d08ae025 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:18.184713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1438/2896] 10.222.0.20 () {58 vars in 1152 bytes} [Tue Jul 12 04:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 04:08:18.188984 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adfa36-031c-4142-9d14-ff1009734549 None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:18.198356 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adfa36-031c-4142-9d14-ff1009734549 None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:18.203388 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcdc79-9ca8-4872-bbca-aa8ca934788b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:18.204045 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcdc79-9ca8-4872-bbca-aa8ca934788b admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.204347 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcdc79-9ca8-4872-bbca-aa8ca934788b admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.204616 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcdc79-9ca8-4872-bbca-aa8ca934788b admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.205312 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcdc79-9ca8-4872-bbca-aa8ca934788b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:18.207208 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcdc79-9ca8-4872-bbca-aa8ca934788b admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:18.236228 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1459/2897] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:08:18 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:08:18.240202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4e6a3-6641-4703-83d0-1aa7d08ae025 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:18.250357 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4e6a3-6641-4703-83d0-1aa7d08ae025 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:18.308988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6073d-c76d-4c71-a60e-788062d7314b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:18.309654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6073d-c76d-4c71-a60e-788062d7314b admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.309936 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6073d-c76d-4c71-a60e-788062d7314b admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.310208 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6073d-c76d-4c71-a60e-788062d7314b admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.310905 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6073d-c76d-4c71-a60e-788062d7314b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:18.312909 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6073d-c76d-4c71-a60e-788062d7314b admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:18.335163 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1439/2898] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:08:18 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:08:18.339509 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcdc79-9ca8-4872-bbca-aa8ca934788b admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:18.349881 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcdc79-9ca8-4872-bbca-aa8ca934788b admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:18.354646 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e942ec-506c-4a1a-9b79-a8ca81fa116b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:18.355308 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e942ec-506c-4a1a-9b79-a8ca81fa116b admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.355592 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e942ec-506c-4a1a-9b79-a8ca81fa116b admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.355867 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e942ec-506c-4a1a-9b79-a8ca81fa116b admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.356659 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e942ec-506c-4a1a-9b79-a8ca81fa116b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:18.358282 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e942ec-506c-4a1a-9b79-a8ca81fa116b admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:18.368905 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1460/2899] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:08:18.372864 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6073d-c76d-4c71-a60e-788062d7314b admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:18.382980 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6073d-c76d-4c71-a60e-788062d7314b admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:18.388035 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010dda1-3a36-45eb-b043-9d3b4d454b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:18.388712 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010dda1-3a36-45eb-b043-9d3b4d454b6f admin admin] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.388991 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010dda1-3a36-45eb-b043-9d3b4d454b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.389266 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010dda1-3a36-45eb-b043-9d3b4d454b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.389988 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010dda1-3a36-45eb-b043-9d3b4d454b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:18.391592 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010dda1-3a36-45eb-b043-9d3b4d454b6f admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:18.402475 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1440/2900] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 04:08:18.407002 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e942ec-506c-4a1a-9b79-a8ca81fa116b admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:18.422824 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e942ec-506c-4a1a-9b79-a8ca81fa116b admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:18.429171 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfc858-3c00-4b23-9756-2f4f79e442cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:18.430054 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfc858-3c00-4b23-9756-2f4f79e442cd admin admin] REQUEST_METHOD: `PUT` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.430337 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfc858-3c00-4b23-9756-2f4f79e442cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.430610 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dfc858-3c00-4b23-9756-2f4f79e442cd admin admin] PATH_INFO: `/v3/projects/715790d79daa428a8a4e422239bd0504/users/e50726eb966a4fafa52074d7dd01f4c3/roles/8f3ea95f038d4d8fbd1fd4897be36344` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.445975 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfc858-3c00-4b23-9756-2f4f79e442cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=715790d79daa428a8a4e422239bd0504, user_id=e50726eb966a4fafa52074d7dd01f4c3, role_id=8f3ea95f038d4d8fbd1fd4897be36344)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:18.448705 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dfc858-3c00-4b23-9756-2f4f79e442cd admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:18.462932 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1461/2901] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:08:18 2022] PUT /identity/v3/projects/715790d79daa428a8a4e422239bd0504/users/e50726eb966a4fafa52074d7dd01f4c3/roles/8f3ea95f038d4d8fbd1fd4897be36344 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:18.473654 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372cd59-af2a-4001-9e31-c77abab457f3 None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.473946 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372cd59-af2a-4001-9e31-c77abab457f3 None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.474227 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372cd59-af2a-4001-9e31-c77abab457f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.505206 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-1372cd59-af2a-4001-9e31-c77abab457f3 None None] MFA Rules not processed for user `e50726eb966a4fafa52074d7dd01f4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:18.506438 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-1372cd59-af2a-4001-9e31-c77abab457f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:18.572848 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1441/2902] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:18.577420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfc858-3c00-4b23-9756-2f4f79e442cd admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:18.636349 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dfc858-3c00-4b23-9756-2f4f79e442cd admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:18.641623 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b3884-57d5-4ba8-888e-5c04aaed94a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:18.642249 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b3884-57d5-4ba8-888e-5c04aaed94a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.642524 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b3884-57d5-4ba8-888e-5c04aaed94a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.642803 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b3884-57d5-4ba8-888e-5c04aaed94a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.643481 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b3884-57d5-4ba8-888e-5c04aaed94a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:18.645858 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b3884-57d5-4ba8-888e-5c04aaed94a2 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:18.673063 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1462/2903] 10.222.0.20 () {60 vars in 1152 bytes} [Tue Jul 12 04:08:18 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:08:18.677262 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372cd59-af2a-4001-9e31-c77abab457f3 None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:18.688503 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372cd59-af2a-4001-9e31-c77abab457f3 None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:18.746415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:18.754524 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df34c9a-9870-4278-9cde-7c8bcbb221f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:18.755444 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df34c9a-9870-4278-9cde-7c8bcbb221f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.755759 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df34c9a-9870-4278-9cde-7c8bcbb221f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.756028 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df34c9a-9870-4278-9cde-7c8bcbb221f9 admin admin] PATH_INFO: `/v3/users` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.756796 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df34c9a-9870-4278-9cde-7c8bcbb221f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:18.759127 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df34c9a-9870-4278-9cde-7c8bcbb221f9 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:18.780263 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1442/2904] 10.222.0.20 () {60 vars in 1143 bytes} [Tue Jul 12 04:08:18 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 04:08:18.784365 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b3884-57d5-4ba8-888e-5c04aaed94a2 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:18.795278 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b3884-57d5-4ba8-888e-5c04aaed94a2 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:18.801275 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b821543-d799-4fd8-8529-46d5298de652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:18.801943 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b821543-d799-4fd8-8529-46d5298de652 admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.802229 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b821543-d799-4fd8-8529-46d5298de652 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.802504 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b821543-d799-4fd8-8529-46d5298de652 admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.803265 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b821543-d799-4fd8-8529-46d5298de652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:18.804933 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b821543-d799-4fd8-8529-46d5298de652 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:18.816044 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1463/2905] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:08:18.820395 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df34c9a-9870-4278-9cde-7c8bcbb221f9 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:18.831439 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df34c9a-9870-4278-9cde-7c8bcbb221f9 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:18.837202 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2e96d-a2f2-43dc-a7e8-dcc14f80b0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:18.838083 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2e96d-a2f2-43dc-a7e8-dcc14f80b0b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.838383 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2e96d-a2f2-43dc-a7e8-dcc14f80b0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.838642 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2e96d-a2f2-43dc-a7e8-dcc14f80b0b7 admin admin] PATH_INFO: `/v3/projects/1cad27c545c245fe87bcc0de7dc2b237/users/da500643b0eb48f7ad1686734b7d679b/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.855032 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2e96d-a2f2-43dc-a7e8-dcc14f80b0b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cad27c545c245fe87bcc0de7dc2b237, user_id=da500643b0eb48f7ad1686734b7d679b, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:18.857731 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2e96d-a2f2-43dc-a7e8-dcc14f80b0b7 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:18.872525 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1443/2906] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:08:18 2022] PUT /identity/v3/projects/1cad27c545c245fe87bcc0de7dc2b237/users/da500643b0eb48f7ad1686734b7d679b/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:18.876594 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b821543-d799-4fd8-8529-46d5298de652 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:18.886953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b821543-d799-4fd8-8529-46d5298de652 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:18.935718 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:18.946299 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ec506-47ad-4de0-a42d-ff47cce7ae6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:18.947640 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ec506-47ad-4de0-a42d-ff47cce7ae6b admin admin] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.948018 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ec506-47ad-4de0-a42d-ff47cce7ae6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.948379 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ec506-47ad-4de0-a42d-ff47cce7ae6b admin admin] PATH_INFO: `/v3/roles` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.949735 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ec506-47ad-4de0-a42d-ff47cce7ae6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:18.952187 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ec506-47ad-4de0-a42d-ff47cce7ae6b admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:18.964254 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1464/2907] 10.222.0.20 () {58 vars in 1121 bytes} [Tue Jul 12 04:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 04:08:18.970640 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2e96d-a2f2-43dc-a7e8-dcc14f80b0b7 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:18.986756 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2e96d-a2f2-43dc-a7e8-dcc14f80b0b7 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:18.992531 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aaf67-61d5-481c-badf-6e41d14cd743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:18.993433 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aaf67-61d5-481c-badf-6e41d14cd743 admin admin] REQUEST_METHOD: `PUT` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:18.993713 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aaf67-61d5-481c-badf-6e41d14cd743 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:18.994002 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46aaf67-61d5-481c-badf-6e41d14cd743 admin admin] PATH_INFO: `/v3/projects/1cad27c545c245fe87bcc0de7dc2b237/users/da500643b0eb48f7ad1686734b7d679b/roles/bef972566e4a4b3b97d844491e12ace2` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:18.996280 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aaf67-61d5-481c-badf-6e41d14cd743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cad27c545c245fe87bcc0de7dc2b237, user_id=da500643b0eb48f7ad1686734b7d679b, role_id=bef972566e4a4b3b97d844491e12ace2)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:18.998992 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46aaf67-61d5-481c-badf-6e41d14cd743 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:19.010408 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1444/2908] 10.222.0.20 () {60 vars in 1482 bytes} [Tue Jul 12 04:08:18 2022] PUT /identity/v3/projects/1cad27c545c245fe87bcc0de7dc2b237/users/da500643b0eb48f7ad1686734b7d679b/roles/bef972566e4a4b3b97d844491e12ace2 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:19.024558 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b918947-d1e4-4e45-bc9d-d1fe7086ab9a None None] REQUEST_METHOD: `POST` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:19.024851 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b918947-d1e4-4e45-bc9d-d1fe7086ab9a None None] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:19.025156 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b918947-d1e4-4e45-bc9d-d1fe7086ab9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:19.053019 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.auth.core [None req-7b918947-d1e4-4e45-bc9d-d1fe7086ab9a None None] MFA Rules not processed for user `da500643b0eb48f7ad1686734b7d679b`. Rule list: `[]` (Enabled: `True`). {{(pid=86935) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:19.054219 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-7b918947-d1e4-4e45-bc9d-d1fe7086ab9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:19.124351 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1465/2909] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:19.135541 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aaf67-61d5-481c-badf-6e41d14cd743 admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:19.137350 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-b46aaf67-61d5-481c-badf-6e41d14cd743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:19.198341 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46aaf67-61d5-481c-badf-6e41d14cd743 admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:19.202194 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ee73b-c3c9-471d-b111-e4c3e36d1315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cee6da907b614dc5b6ffb47e2860f459', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:19.202836 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ee73b-c3c9-471d-b111-e4c3e36d1315 service cinder] REQUEST_METHOD: `GET` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:19.203116 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ee73b-c3c9-471d-b111-e4c3e36d1315 service cinder] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:19.203385 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ee73b-c3c9-471d-b111-e4c3e36d1315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:19.262591 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ee73b-c3c9-471d-b111-e4c3e36d1315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:19.264049 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ee73b-c3c9-471d-b111-e4c3e36d1315 service cinder] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:19.273639 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1445/2910] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:08:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:08:21.826949 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b918947-d1e4-4e45-bc9d-d1fe7086ab9a None None] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:21.899220 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b918947-d1e4-4e45-bc9d-d1fe7086ab9a None None] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:21.904815 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:21.905723 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:21.906016 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:21.906287 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] PATH_INFO: `/v3/users/e50726eb966a4fafa52074d7dd01f4c3` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:21.907526 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e50726eb966a4fafa52074d7dd01f4c3)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:21.909557 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:21.963838 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e50726eb966a4fafa52074d7dd01f4c3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:21.964601 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e50726eb966a4fafa52074d7dd01f4c3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:21.973109 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e50726eb966a4fafa52074d7dd01f4c3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:21.980445 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e50726eb966a4fafa52074d7dd01f4c3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:21.990361 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:21.992249 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e50726eb966a4fafa52074d7dd01f4c3'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:21.999261 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1466/2911] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:08:21 2022] DELETE /identity/v3/users/e50726eb966a4fafa52074d7dd01f4c3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:22.003970 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc2bde-1c9e-40ce-b759-32bd2eea481d None None] REQUEST_METHOD: `POST` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:22.004582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc2bde-1c9e-40ce-b759-32bd2eea481d None None] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:22.004582 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc2bde-1c9e-40ce-b759-32bd2eea481d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:22.022126 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.auth.core [None req-fddc2bde-1c9e-40ce-b759-32bd2eea481d None None] MFA Rules not processed for user `819f4355d22549de99860edd3df36888`. Rule list: `[]` (Enabled: `True`). {{(pid=86934) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 04:08:22.023331 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fddc2bde-1c9e-40ce-b759-32bd2eea481d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:22.083775 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1446/2912] 10.222.0.20 () {58 vars in 957 bytes} [Tue Jul 12 04:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 04:08:22.095073 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:22.096775 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:22.157462 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cadbd7-b5cc-4fc8-b8e8-8f8c6771fc62 admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:22.162430 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b16392-367a-4974-b8f0-e7c04a7151b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '139be21e7f644b5796d30158851ebd00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:22.163116 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b16392-367a-4974-b8f0-e7c04a7151b0 service neutron] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:22.163400 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b16392-367a-4974-b8f0-e7c04a7151b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:22.163679 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b16392-367a-4974-b8f0-e7c04a7151b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:22.174067 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b16392-367a-4974-b8f0-e7c04a7151b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:22.175525 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b16392-367a-4974-b8f0-e7c04a7151b0 service neutron] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:22.185048 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1467/2913] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:08:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:08:22.460080 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc2bde-1c9e-40ce-b759-32bd2eea481d None None] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:22.461600 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-fddc2bde-1c9e-40ce-b759-32bd2eea481d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:22.470192 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:22.484920 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc2bde-1c9e-40ce-b759-32bd2eea481d None None] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:22.490689 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:22.491567 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:22.491843 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:22.492157 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] PATH_INFO: `/v3/projects/715790d79daa428a8a4e422239bd0504` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:22.493486 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=715790d79daa428a8a4e422239bd0504)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:22.495415 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:22.544472 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '715790d79daa428a8a4e422239bd0504', 'request_id': 'req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:22.553303 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '715790d79daa428a8a4e422239bd0504', 'request_id': 'req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:22.559302 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '715790d79daa428a8a4e422239bd0504', 'request_id': 'req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:22.570752 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] The token cache is being invalidate because project 715790d79daa428a8a4e422239bd0504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:08:22.571043 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:22.571684 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:22.573267 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1447/2914] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:08:22 2022] DELETE /identity/v3/projects/715790d79daa428a8a4e422239bd0504 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:22.577404 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b16392-367a-4974-b8f0-e7c04a7151b0 service neutron] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:22.587890 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b16392-367a-4974-b8f0-e7c04a7151b0 service neutron] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:22.650217 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:22.651108 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:22.651426 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:22.651756 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] PATH_INFO: `/v3/users/da500643b0eb48f7ad1686734b7d679b` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:22.653106 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] RBAC: Authorizing `identity:delete_user(user_id=da500643b0eb48f7ad1686734b7d679b)` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:22.655046 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:22.702050 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da500643b0eb48f7ad1686734b7d679b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:22.702716 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da500643b0eb48f7ad1686734b7d679b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:22.711363 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da500643b0eb48f7ad1686734b7d679b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:22.718196 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da500643b0eb48f7ad1686734b7d679b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:22.729787 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.notifications [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da500643b0eb48f7ad1686734b7d679b'} {{(pid=86935) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:22.735953 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1468/2915] 10.222.0.20 () {58 vars in 1223 bytes} [Tue Jul 12 04:08:22 2022] DELETE /identity/v3/users/da500643b0eb48f7ad1686734b7d679b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:23.008176 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] Authenticating user token {{(pid=86934) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:08:23.009703 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.fernet_utils [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86934) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:08:23.025529 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db14630-d2ae-40be-a5dd-eb5dc28bdbbf admin admin] Validating token access rules against request {{(pid=86934) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:08:23.084906 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3181e07-c6ea-4670-9144-cda4e739d800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '819f4355d22549de99860edd3df36888', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '579b7a08aae94d3fa59617c603862f30', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86934) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:08:23.085790 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3181e07-c6ea-4670-9144-cda4e739d800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:08:23.086079 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3181e07-c6ea-4670-9144-cda4e739d800 admin admin] SCRIPT_NAME: `/identity` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:08:23.086354 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3181e07-c6ea-4670-9144-cda4e739d800 admin admin] PATH_INFO: `/v3/projects/1cad27c545c245fe87bcc0de7dc2b237` {{(pid=86934) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:08:23.087630 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3181e07-c6ea-4670-9144-cda4e739d800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cad27c545c245fe87bcc0de7dc2b237)` {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:08:23.089634 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3181e07-c6ea-4670-9144-cda4e739d800 admin admin] RBAC: Authorization granted {{(pid=86934) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:08:23.141054 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a3181e07-c6ea-4670-9144-cda4e739d800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cad27c545c245fe87bcc0de7dc2b237', 'request_id': 'req-a3181e07-c6ea-4670-9144-cda4e739d800'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:23.150056 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a3181e07-c6ea-4670-9144-cda4e739d800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cad27c545c245fe87bcc0de7dc2b237', 'request_id': 'req-a3181e07-c6ea-4670-9144-cda4e739d800'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:23.155908 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a3181e07-c6ea-4670-9144-cda4e739d800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cad27c545c245fe87bcc0de7dc2b237', 'request_id': 'req-a3181e07-c6ea-4670-9144-cda4e739d800'} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:23.167051 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a3181e07-c6ea-4670-9144-cda4e739d800 admin admin] The token cache is being invalidate because project 1cad27c545c245fe87bcc0de7dc2b237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86934) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 04:08:23.167358 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a3181e07-c6ea-4670-9144-cda4e739d800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:23.168030 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG keystone.notifications [None req-a3181e07-c6ea-4670-9144-cda4e739d800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86934) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 04:08:23.169604 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: [pid: 86934|app: 0|req: 1448/2916] 10.222.0.20 () {58 vars in 1232 bytes} [Tue Jul 12 04:08:23 2022] DELETE /identity/v3/projects/1cad27c545c245fe87bcc0de7dc2b237 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 04:08:32.732555 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:DELETE=11,keystone:SELECT=81,keystone:INSERT=1 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:08:33.164256 dv-cinder-847730-4-iscsi devstack@keystone.service[86934]: DEBUG dbcounter [-] [86934] Writing DB stats keystone:SELECT=76,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86934) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 04:13:35.604506 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] Authenticating user token {{(pid=86935) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 04:13:35.606964 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:13:35.683971 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6cb28-cd68-4ac6-a53c-87038aece7db admin admin] Validating token access rules against request {{(pid=86935) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 04:13:35.745441 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106dc545-4269-4734-8fa6-bfe8f781bae0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eded06390a2744c9aa52f77e3bfc2db9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '48b17053daa84e4fa94840777cb1bee6', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86935) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 04:13:35.746095 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dc545-4269-4734-8fa6-bfe8f781bae0 service placement] REQUEST_METHOD: `GET` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 04:13:35.746378 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dc545-4269-4734-8fa6-bfe8f781bae0 service placement] SCRIPT_NAME: `/identity` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 04:13:35.746677 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dc545-4269-4734-8fa6-bfe8f781bae0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86935) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 04:13:35.748773 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.fernet_utils [None req-106dc545-4269-4734-8fa6-bfe8f781bae0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86935) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 04:13:35.760809 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106dc545-4269-4734-8fa6-bfe8f781bae0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 04:13:35.762420 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106dc545-4269-4734-8fa6-bfe8f781bae0 service placement] RBAC: Authorization granted {{(pid=86935) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 04:13:35.772488 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: [pid: 86935|app: 0|req: 1469/2917] 10.222.0.20 () {60 vars in 1365 bytes} [Tue Jul 12 04:13:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 04:13:45.768476 dv-cinder-847730-4-iscsi devstack@keystone.service[86935]: DEBUG dbcounter [-] [86935] Writing DB stats keystone:SELECT=58 {{(pid=86935) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}